Hacker News Reader

Comment by amluto | Hacker News Reader

The institutional trading world uses a lot of private network connections, and it would be amazing if the whole industry migrated to IPv6. Then everyone could have a real subnet for each link with genuine globally-unique addresses. It wouldn’t even be a hard migration — vendors force everyone to update their network config on a semi-regular basis anyway.

View on HN

For institutions on the leading edge, the extra bytes incurred by the IPV6 ethernet frame make it a non starter. 160 bits for IPV4 vs 320bits for IPV6 @ 10GHz is a 16 nanosecond penalty on every message you send.

With those sorts of requirements, it seems like firms would find value in using a different layer-3 protocol (perhaps even custom) to reduce that overhead further.

Yep, the microwave links are often using custom layer 3 tech (1 byte messages!).

Beyond that, you need to vertically integrate a hardware shop and get custom layer 3 hardware, otherwise you're not going to beat off the shelf Arista/Cisco switches doing IPv4 routing.

Also, at the end of the day, you need to be shipping your bytes to an external consumer (e.g. an exchange), so you're probably going to need to have an IPv? layer at some point anyways.

It's crazy the lengths to which trading firms are going to squeeze nanoseconds out of latency. It makes me wonder what value we're getting, from a societal perspective.

Better network tech =). You can probably thank trading for consumer 10G now and 40G in ~5 years.

I doubt that’s true. HFT is big business but nowhere near that big. Keep in mind that a lot of the big HFT players failed a few years back due to revenues being too small to support the R&D operations. I think that what really made 10G affordable is trickle-down from the SAN world.

Yeah you're right, above is a bit sensationalized. I think it still probably had a (positive) impact on pricing and time to market though

Yea you can pick up a PCIe 10G adapter for $40 on Amazon and a 4 port 10gb dlink switch for $200

It's all about finding and capitalizing on arbitrages in the market the popup and disappear all the time in the blink of an eye.

I suspect this makes no difference whatsoever as long as everyone is forcibly migrated at once.

You can only forcibly migrate the edge protocol. There will still be competition for small ethernet frames to execute against geo arbitrage opportunities.

What you'll (likely) see is internal networks on V4 and V4->V6 translation switches at the edge so you only pay the 16ns cost on ingress/egress.

Honest question, not trying to incite unrest: Why does IPv6 even matter for private networks? My understanding is the main (only?) benefit of IPv6 is the vastly increased address space, are there that many private networks in existence that exhaust all IPv4 private network IP ranges?

> are there that many private networks in existence that exhaust all IPv4 private network IP ranges?

No, but there are many private networks in existence that use overlapping subsets of the private network ip ranges.

I bet there are. It only takes ~16M residential NAT “routers” to hit 2^32 total addresses. And there are way more than 2^32 things on the Internet.

it's much more simple than that.

While there are 16,777,216 available addresses under 10.0.0.0/8, most organizations will do the reasonable thing of allocating nice regular subnets for each area and use case, so you can easily end up with one org using things like

  10.0.0.0/16 for infra
  10.8.0.0/16 for allocating end user /24s
  10.16.0.0/16 for whatever

and then that org will merge with another who as it turns out did exactly the same thing, and now the internal subnets all overlap. Even though each company may only be using a few thousand addresses, the address space itself overlaps.

Even if you can plan things ahead of time and divided the /8 into 32 /13s or something and give each sub company 500k addresses, you'd still be screwed on a merger.

Even with private networks, having plenty of address bits makes things a lot easier.

My employer's internal network includes hosts with addresses in the 10.0.0.0/8 private address range. So does my home network. I have to be careful when choosing subnets, or else things will break in weird ways when I connect to my work VPN. With IPv6, I don't have to worry about this because every machine can be assumed to have a globally-unique address, even if that address isn't normally exposed to the outside world. And more importantly, it's easy to look at the address prefix and figure out where to route packets.

Ah, that's actually something I have hit (a lot.. my employer's and home's networks also clash), but I didn't realize IPv6 would solve it until now!

The right description for most networks in finance isn't really private, but rather semi-public. NYSE's matching engines aren't routable over the public internet, but they'll give access to anybody willing to pay the price. Same goes for any of the hundreds of other exchanges out there. And lots of broker/dealers need to talk to multiple different exchanges, as well as to each other. It would be nice to do all of that communication using one globally unique IP address, rather than with a rats nest of NAT for each gateway.

I’ve seen traceroutes on private network setups that involve a catastrophe of different conflicting uses of 10.whatever addresses. Also, people use NAT on some of these private network for reasons that escape me.

Getting everyone on the same address namespace would be a big simplification.

Reply on news.ycombinator.com