Scriptable GeoDNS: Leaving AWS for Bunny

I liked it but, it was a hassle to get it running.

For one, your ability to control traffic flow is limited to very coarse knobs, since where your traffic lands is dependent entirely on the BGP routes you advertise and who you advertise them to.

The basic idea of BGP is that you advertise the same IP blocks to multiple peers, and then every other network will choose the destination that is the least number of ‘hops’ away. However, hops are not always a perfect representation of the fastest, or even closest, destination. Sometimes a hop travels a far distance, or is over a slow or congested link. Sure, one connection might be 2 hops instead of 3, but if those two hops are slow the route with “more” hops might actually be faster.

You can solve this with a lot of different tricks, such as adding extra fake hops to your announcements in certain places, but you still have to be constantly monitoring the route traffic is taking to make sure your traffic continues to take the best route.

Secondly, there is no easy way to distribute traffic from a single network to multiple destinations for load balancing purposes. Every request coming from a particular network is going to make the same anycast calculation, so it will send all requests to the same destination. If you want to split that up into two different destinations, you have to go back to using DNS tricks or using different destinations IPs to split traffic up.

We would do this sort of load management by dividing traffic onto different IP blocks, and then dynamically advertising or not advertising those blocks from datacenters depending on load. If one datacenter is becoming overloaded, you pull some blocks from the advertisements for that datacenter, and that traffic stops being routed there.

There are a ton of other factors that go into it, but the bottom line is that anycast is not a simple fix to this problem.