Comment by montasaurus | Hacker News Reader

Comment by montasaurus | Hacker News Reader

Parent Full thread

The fact that you can reach almost any IP address is kind of a small miracle when you start digging into it. There are a bunch of interesting examples of addresses becoming unreachable or sent to the wrong destination, both accidentally and intentionally.

The BGP protocol is how each network announces what IP addresses can be reached through them. Those announcements can often be faked, in a process called BGP Hijacking. This happened to EtherWallet: https://www.theverge.com/2018/4/24/17275982/myetherwallet-ha...

Some address ranges are reserved for internal use within each network. If the network is big enough, some network operators "borrow" other less-used public IP addresses and re-purpose them for internal use. This means that traffic inside that network or transiting through it can't reach those IP addresses. Cloudflare's 1.1.1.1 DNS server (and 1.0.0.0/8 more generally) are affected by that a lot: https://blog.cloudflare.com/fixing-reachability-to-1-1-1-1-g...

Also, sometimes single point of failure connections just...break. It's not guaranteed that every connection is redundant, and it's definitely possible for chunks of the internet to just be "disconnected" from the rest.

In researching a report about using ping services to locate VPN servers, I found a few examples where nearby ISPs only peer remotely. Iceland via Germany. Zurich via Italy and Germany. Salt Lake City via Boulder, LA, etc.

"only" as in trying to avoid local peering? Do you have an idea why they might do that?

Yes, no local peering. I have no clue why.

The Iceland example: https://keybase.pub/mirimir/IVPN-is1.gw.ivpn.net-All-Probes-...

From the IVPN article:

> Most notably, the lowest-rtt probe for IVPN server is1.gw.ivpn.net is in Amsterdam, NL. The data is somewhat “V” shaped, with the lowest minimum rtt at ~2,000 km. And indeed, the distance between Reykjavik, IS and Amsterdam, NL is 2013 km. However, given my long-term working relationship with IVPN, one of their network engineers verified that this server is indeed in Reykjavik, IS. It’s also unlikely that the maplatency.com probe “IS midlar ehf” is actually in Amsterdam, because that’s an AS in Iceland. It’s arguably most likely that the probe (in Iceland AS60300) and is1.gw.ivpn.net (in Iceland AS44515) just weren’t peering directly, but instead through an AS near Amsterdam.

My guess is because it does cost some money to do local peering. My understanding is you need to contact each individual company you want to peer with, make sure you both have a Point of Presence in the same datacenter, sign a contract, etc. So you both need to be renting space in that building and both have an open port on your router, and then you need to pay the datacenter to actually physically run the wire between the routers.

Thanks. I hadn't thought it through so clearly.

But wait, how does traffic get to that remote peer?

> how does traffic get to that remote peer?

Not sure what you mean. The two networks (say, a DSL and Cable provider in the same area), need to physically connect at some location in the area (usually at a "Carrier Hotel"). Once they're connected, traffic originating from a DSL customer is routed through the DSL network to the Carrier Hotel, then across the to the Cable provider's network, and then through that Cable network to the Cable customer.

For example, I live in South Bend, IN, where we have Comcast Cable and AT&T DSL. Although we have a few places in town where Comcast and AT&T _could_ each have a point of presence, and then peer with each other, they don't think that's worth it. Instead, traffic is routed to the nearest peering point, which in my case is 350 E Cermak in Chicago, 100 miles away. Peering locally would probably save about 10ms, but that's probably not really worth it.

OK, I get it. In your example, AT&T and Comcast peer in Chicago. In my example, the two Icelandic ASNs peered near Amsterdam. Likely through international intermediaries.

It means that the only people that can cut your connectivity off are in a completely different jurisdiction. Useful if you have a lot of sketchy traffic.

But isn't there always some intermediary with a local presence?

Reply on news.ycombinator.com