AS215855, two years later

In March 2024, I wrote about getting my own ASN. I had found a sponsor, got some IPv6 space, brought up BGP on a couple of Debian machines, and started announcing routes.

That was AS215855.

More than two years later, I am still doing it. The network has changed, I have changed routers a few times, and I have written rather more software around it than I expected to.

I work with networks all day. Apparently that still leaves a few things I want to try in the evening.

Getting the ASN was the beginning

The original setup was fairly straightforward. Securebit sponsored the resources, I got a /44 of IPv6 space, and I used Debian and BIRD2 to connect locations in Europe and the US. I split the address space into smaller prefixes, signed them with RPKI, and worked out how to get traffic to the right place.

There was a lot of satisfaction in seeing my own ASN show up in the routing table. You configure something on a VPS, look at it from somewhere else on the Internet, and there it is. A route you originated.

The topology from my first post in March 2024. This is where it started, rather than a diagram of the network today.

Getting that first announcement working gave me a reason to keep going. Once I had address space and routers in different places, I could actually use them. Then I had to decide how they should connect, which paths I preferred, what should happen when one failed, and how I would know something had gone wrong.

The setup had become something I needed to operate.

BIRD stayed. The router at home changed.

Linux and BIRD were a good place to start. I could express routing policy in a configuration file, manage it with Ansible, and inspect what was happening with tools I already understood.

BIRD has remained part of the network. At home, I got curious about other options.

That led to the UDM-Pro, a used MikroTik CCR2004, and more switching around than I originally intended. I have already written about both, so I will spare you another full comparison here.

The useful part of trying the MikroTik was having to implement familiar ideas in a different system. I knew what I wanted the routing policy to do. Working out how to express it in RouterOS took longer.

Under BIRD, I was used to writing filters as blocks of logic in a config file. RouterOS had its own syntax, rule handling, and ways of inspecting the result. The concepts carried over. My muscle memory did not.

It also affected the things around the router. Anything I had built to write a BIRD configuration or run birdc needed another look.

I had managed to give myself a small migration project. At home. For fun, apparently.

A few milestones from the earlier posts. The tools grew alongside the network.

I ended up building the monitoring I wanted

One of the bigger things to come out of AS215855 has been NetPulse.

It started because I wanted something nicer than MRTG for interface traffic graphs. A Go backend, a React frontend, and some SNMP polling seemed like a reasonable project.

Then I wanted more information from the Linux routers. That led to agents that could query BIRD directly. BGP session monitoring followed, along with flow analysis, alerts, and a looking glass.

There was no shortage of things to add once I was using it against my own network. I would be investigating something, find myself missing a piece of information, and have another feature to work on.

That is a fairly effective way to lose control of the scope.

AI helped me build a lot of it. I have been open about that in the earlier posts. I could describe what I wanted, work through an implementation, run it against the network, and find out where it fell apart. That made projects practical that I probably would not have finished by writing every part myself.

It also gave me another set of things to maintain. The collector, the agents, the database, the frontend, the update mechanism. A nicer traffic graph had acquired quite a few dependencies.

A BGP session can be perfectly happy while I am not

Lagbuster came out of a different problem.

I had paths that could become slow while their BGP sessions stayed established. Routes were still there and traffic still flowed, but the experience had become noticeably worse. I could notice it and change the routing preference by hand, or leave it alone.

I wanted to automate that decision.

Lagbuster measured latency to the edge routers, compared each path against its own baseline, and changed the preference when a path had degraded enough to justify it.

The measurements were the easy part. The harder part was deciding when to act, and when to leave things alone. A short spike should not send traffic bouncing between paths. A path that had just recovered needed time to prove it was healthy before getting traffic back.

The decision logic stays separate from the code that applies the routing change. That made it possible to carry it from BIRD over to RouterOS.

Then I replaced the BIRD router at home with the MikroTik, and the part of Lagbuster that applied those decisions no longer worked.

A later evening of bad latency to Ashburn gave me a reason to add the RouterOS backend. The decision logic could stay; the code that changed the router needed replacing.

That was a useful test of the design. I could carry the behavior I wanted across to different equipment without starting over.

The less glamorous lessons tend to stick

Some of the most useful learning has come from ordinary maintenance going wrong.

A RouterOS upgrade once left my Comcast connection unable to get a DHCP lease. There were reports of trouble in the building, modem firmware had entered the conversation, and I spent a good while convinced I was dealing with an ISP problem.

A laptop connected directly to the gateway worked. The MikroTik could also get a lease from a Linux DHCP server. The failure was in how the two behaved together after the upgrade.

I eventually traced it to IPv6 Router Advertisements leaving the WAN interface.

That incident took more troubleshooting than bringing up some of the BGP sessions in the first place. It also left me with a much better understanding of that part of my own configuration.

There have been smaller lessons too. Publishing a geofeed involved checking how the file was served. Adding prefix-length statistics to BIRD Exporter meant looking more closely at what was actually in a routing table.

Those jobs rarely sound exciting beforehand. They are often the ones that make me understand the setup properly.

What I would tell myself at the start

I would still start with Linux and BIRD. It gave me a setup I could read, change, and reproduce without needing much equipment.

I would also spend more time documenting the network while it was small. Which prefixes belonged where, why a filter existed, how the tunnels fitted together, and what I expected to happen during a failure. Those details seem obvious when you have just configured them. They become less obvious after a few more experiments.

For automation, I would start with a narrow job and make the decisions visible. Lagbuster changing routing preference is useful to me because I can look back at the measurements and see why it acted. I want that same ability when something behaves unexpectedly.

And I would keep writing things down. Several posts on this blog exist because I spent an evening debugging something and wanted a record of the answer. They are useful to me even if nobody else ever hits the same problem.

Still running it

AS215855 has given me a place to try things I would not casually change at work. Different routers, routing policies, monitoring ideas, and software that starts with one fairly small problem.

Some experiments have stayed. Others have sent me back to a previous setup. Along the way, I have ended up with a network I use, a collection of tools around it, and a growing list of things I would like to improve.

When I got the ASN, I wanted somewhere to do networking on my own terms.

More than two years later, that is still a pretty good reason to keep it running.