40% of Google users now connect via IPv6
google.com
google.com
On Starlink it's impossible to host a server socket directly, which makes any peer to peer networking a PITA. Geocoding IP addresses doesn't work so I have to bend over backwards to convince, say, Youtube TV that I'm in the Sacramento metro and not LA where the POP is. Also the shared IP addresses seem to trip a lot of DDOS protection; I fill out 10x as many CAPTCHAs on Starlink as I do on my other ISP. And I sometimes get random network stability problems; a few weeks ago Starlink screwed something up so no one could keep a persistent connection up more than a few minutes. Seems to be fixed now, but I bet it was their CGNAT system.
I realize half the world lives with CGNAT. It's not unusable, at least web browsing works more or less. But IPv6 would solve all these problems. A little surprised that a new ISP created in 2021 wouldn't have IPv6 support as one of their launch features. There's hints they are trying to get it working but it's not an official thing now. Some discussion: https://www.reddit.com/r/Starlink/comments/tjr90n/starlink_i...
This a feature, not a bug. You can now identify a traffic light on a subconscious level 3x further away than the average driver that has an assigned IPv4.
Where average drivers hesitate upon seeing a yellow light, wondering if they have enough time to go or not, you just know that a slight uptick in speed will get you through the intersection right as that yellow flashes over to red.
https://www.tesla.com/careers/search/job/data-annotation-spe...
But, I would be surprised if they don't also use 3rd party labelled data sets.
The problem is with end user devices that do not use a stable DUID and when the client hits the reset button, it changes. We are probably going to work around this by responding from the closest hop and taking the MAC into account.
And connection tracking gets expensive at scale.
But CPU/memory is going down in cost way faster than bandwidth demand is increasing.
Regardless, its way way cheaper than buying IPv4 blocks clearly, otherwise people wouldn't be doing it.
Edit: ok, the problem isn't hardware, it's comedy license fees. https://itprice.com/juniper-price-list/cgn.html
$470k for a license to do CGNAT at 100gbit/sec. Surely these guys are opening themselves up to be replaced with some cheaper open source based software solution?
Good. CGNAT needs to die. Addressing is fundamental and customers deserve not just an address, but their own RANGE, especially now that it's feasible.
Having said that, I’m often equally baffled at just how expensive modern networking hardware is, but as it’s pretty much all of these carrier grade networking solutions being this expensive, I’m assuming it’s somewhat justified.
That doesn’t take away the fact that NAT just adds an expensive layer of complexity on top of it, and I can imagine that in the long term, IPv6 is starting to become much more attractive.
And you can charge them an extra $10 per month for 'pro' internet and let them have 1000 connections for 'all the family'.
But that's besides the point. Your home router can easily have millions of connections open (if they didn't skimp on the ram anyway), but if you have CGNAT boxes that do the same for tens of thousands of customers you also have to take into account that they have to move a lot of traffic. This means routing and doing NAT in software won't cut it anymore, but you need dedicated hardware coupled with very fast specialized memory to handle that traffic.
Also, in some countries ISP's need to map the use of a specific ip address to a specific subscriber for law enforcment purposes. GCNAT is no exception to this and creates a large amount of overhead because the public IPV4 prefix space is shared between multiple customers.
But it’s a solved problem with mature solutions, decades old. Is it really financially expensive?
For instance, most Core/Edge routers (my experience is mainly with juniper MX series, but i assume the model is roughly the same for other vendors), you need specific licenses or interface card's to do stateful services like NAT.
Compared to doing IPv6, which is "just forwarding packets" and doesn't require the hardware to track state in nearly the same manner.
Most serious core/edge hardware vendors also do not put IPv6 behind licenses compared to CGNAT and other NAT-like features, because packet based forwarding is the most basic functionality a router should provide.
Routers which are able to do less state, also are frequently far less expensive.
Worse, they had it and turned it off at some point!
Unfortunately Sony doesn't support IPv6 either.
Without the laser links Starlink also never got the latency advantages it was supposed to have for long range gaming (like US to Europe). Instead it goes down to a basestation and then through traditional means, but that may change with the new bigger satellites if the laser part works this time.
Bugger this. I've had enough of the captcha storm.
I wonder if they’ll go IPv6 once they are doing inter-satellite routing?
with 4G/5G home internet, there's no real point anymore.
Many people do not have this option.
For me personally, the pathing of 5G and my broadband are too similar, so Starlink acts as a redundancy for these.
Or I can use Starlink. I get a solid reliable signal, anywhere from 20-250 Mbps with 60ms latency. It costs $110 for unlimited bandwidth.
The real competition where I am is fixed wireless. That's 12 Mbps, 70ms latency, and $100/mo.
I don't see how 4G/5G home internet existing removes the point of it.
> Removing any unnecessary feature is a must in order to reduce risk.
You can’t scale a space based planetary ISP without IPv6, this isn’t a feature, it’s a requirement.
TL;DR bigger fishes to fry ATM - yes you need IPv6 to scale, no you don't need it right now.
When the ipv6 spec was released, the latest python release did not yet support list comprehensions.
In other words, there is no reason to not support ipv6 out of the box in 2022.
Yes we do. IPv4 exhaustion is a thing.
How?
IPv6 isn't a substitution for CGNAT, it's an addition to it. You either have to keep CGNAT or replace it with dedicated IPv4 for each customer. Dedicated IPv4 is most likely too costly given the limited availability. SpaceX is also trying to cut cost aggressively.
Specifically, if a plain router stops working, BGP will route around it, and all you did was drop packets. If NAT stops working, you don’t just drop packets, you drop whole connections. Applications don’t tend to tolerate dropped connections as well as they tolerate dropped packets.
That is the biggest problem with IPv6. Who is gonna be the first ISP to shut off their IPv4 stack? There is always gonna be some random website that is IPv4.
This gets you off their IP, and also has the added benefit of not letting them analyze your traffic.
It would be nice if there were some way to decouple connectivity from addressing, without becoming a second-class citizen of the internet.
Incidentally this also makes torrenting safe, and everything on Netflix can be downloaded via BitTorrent.
Now it's mostly businesses that are still not using IPv6.
[1]: https://www.internetsociety.org/resources/deploy360/2014/cas...
Found that out the hard way the other week when I went to order a build server without an IPv4.
To add, if they're moving to Azure this might make IPv6 deployment complicated: https://news.ycombinator.com/item?id=29327773
Go is also basically bound to IPv4 since it depends on github to pull packages
its time to make hall of shame for software that does not allow developers to build IPv6 only going forward.
I'm 99% certain that they turn off IPv6 to avoid complaints. Implementing IPv6 on a frontend load balancer is a trivial networking change. But the only way to ensure an IPv6 connection works is for the user's OS, networking, firewall, router, modem, ISP backend network, ISP DNS resolver, target website DNS, and target website load balancer & firewall, all have IPv6 configured properly. If a single step is misconfigured, or uses IPv6 tunneling/translation, every request might be blocked until the website disables IPv6. So don't support IPv6 at all and you avoid headaches.
Avoiding an unnecessary support headache is the basic reason why IPv6 has existed for 26 years and yet Google can still barely get 40% use for its own website. Everybody loves to design a spaceship, but nobody designs for moving between spaceships mid-launch.
Is this also true for the Go module proxy?
$ dig +short AAAA proxy.golang.org
2a00:1450:4009:820::2011My development is completely user driven.
I get hundreds of feature requests and other emails regarding my site per month.
So far, not a single user mentioned IPv6.
So I never added an AAAA record.
The codebase has grown for over 10 years now.
God knows what subtle breakages would happen when IPv6 requests come in.
I wonder if I ever will add IPv6 or just leave the site running on IPv4 forever.
IPv4 servers are reachable by IPv6-only clients. ipv6test.google.com famously has only an A (IPv4) record for a reason.
> God knows what subtle breakages would happen when IPv6 requests come in.
Unless your product deals with the nitty gritty of the networking stack itself, pretty sure everything continues to hum along just fine, if the basics are covered.
pretty sure everything continues
to hum along just fine
Have you ever worked with a live system that grew for 10 years?
There could be tons of IP specific things in there. To enforce rate limits. For security. To do statistics. To do A/B tests...
So far, nobody said IPv4 will be turned off.
Maybe it will work for another 1000 years?
Railroads have been around for over 2000 years and are still going strong. Despite cars and planes.
Postal service has been going on for 4000 years. Since the times of the pharaohs! And It's still going strong, too. Despite phone, email and WhatsApp.
Rail is vastly superior to both planes and cars in many circumstances, if done right.
> Postal service has been going on for 4000 years. Since the times of the pharaohs! And It's still going strong, too.
With significant upgrades to both the transport and addressing layers :)
As a provider, the main benefit I've seen is that every user has a roughly unique IP. It's easier to audit things. It's really messy when lots of users are behind CGNat. Another benefit, eventually, is the cost of IPv4 space (but admittedly not a big problem now).
It's less likely that you will do so at a "customer" request, rather it will be ISP/Hosting provider that will start to charge you evermore increasing fees to rent your IP address. The the Ipv4 space gets more competitive you will see the fees for routable IPv4 address go up; and conveniently there will be "discounts" to go ipv6. That's when I imagine most businesses will make the switch.
How does this work?
In practice DNS64 is now being removed from a majority of networks because some specialty applications (I say "specialty" but these are work VPNs, conference systems and the like) reacts badly (because usually they can't understand IPv6 in the first place), replaced by either DS-Lite or plain dual-stack (possibly with CGNAT for IPv4).
Hetzner and other providers of this size already support hosting options for IPv6.
That said - my home ISP is not providing IPv6 at all, ISP we were getting office connection also provided no IPv6.
If your site is largely centrally hosted and doesn't require many IPs then nobody, including you, is going to care if it's v4 or v6 until ISPs stop providing v4 gateway services in the far far future as whether your site supports IPv4 isn't going to change anything for anyone. Well, unless your site is like the HN crowd where hacking for hacking's sake is the point in which case that's actually odd you haven't received an email.
Google is obviously the exact opposite of this use case hence their interest and monitoring of IPv6 support and the latency impact numbers.
That's because they don't know your website exists.
As long as there is sufficient penetration of these black boxes, virtually everything should be able to talk to everything over v6, and the v4 shim can be removed.
I imagine this black box could be a relatively inexpensive ASIC or FPGA that could be a stand-alone widget, baked into hardware network adapters, or just built into routers, middleboxes, etc.
It is easy to make an IPv4 server accept IPv6 connections. It is relatively difficult to make an IPv4 client connect to IPv6 servers, because there aren't enough bits in the 'destination' field.
It's not that simple:
- The box has to translate IPv6 address space into IPv4 address space, but it's too big to fit. So the box has to be some kind of stateful reverse NAT, with all the problems that that involves, and the hardware requirements go way up.
- The IPv4-only host might make all sorts of assumptions about IPv4 addresses that are no longer valid. E.g. it might cut off addresses that it detects an attack from - but now as soon as two IPv6 addresess get mapped to the same IPv4 address you're going to block a legitimate user (in fact, since changing IPv6 address is easy, you're probably going to pretty quickly block the whole internet). E.g. it might expect to use an IPv4 geoIP database. E.g. it might be speaking a protocol like FTP where it's supposed to make an outbound connection to the client, so now your middlebox has to not only keep track of TCP streams but also the details of every protocol you want to be able to support.
Of course, spinning up a cloud VM is an easy solution these days, but I was surprised at how poor the IPv6 adoption was where I live.
I used to go out of my way to get ipv6 working (back when it was "new" I used a bridge service to get access), but I simply do not have the time to figure out what might be wrong with it anymore.
This usually isn't a problem for power users (who know how to set up and (re)confgure a firewall) nor for most basic users (windows firewall does that for them), but people "in the middle", who install some service and just fully disable the OS's firewall to be able to connect to it, are now vulnerable.
This is not true. Firewall rules work exactly the same in IPv6 land as they do in IPv4 land.
> you need to pass a bunch of ICMPv6 through, to make it work
Indeed you shouldn't block ICMPv6, but that is not really making anything "more complex".
> some residential routers have very bad or even zero firewall support for ipv6
Is there a proven set of routers that go through the trouble of supporting IPv6 routing but not include a firewall?
Yes, rules do work exactly the same, but with IPv4, you just let all the connections out through, and let just the established and connected ones back.
> Indeed you shouldn't block ICMPv6, but that is not really making anything "more complex".
But it is... you need a bunch of new rules to pass through, limit or block a bunch of ICMPv6 messages.. there's a whole RFC just for that - https://datatracker.ietf.org/doc/html/rfc4890
> Is there a proven set of routers that go through the trouble of supporting IPv6 routing but not include a firewall?
Yeah, a bunch of ISP CPEs have just a single checkmark "IPv6 firewall" on/off, and some older ones not even that (i'm talking about old sagem and innbox equipment i came in contact with, not sure about other telcos and the shity cpes they give out to the customers).
The same is typically true of IPv6 for default configurations. You aren’t required to allow IPv6 hosts to accept unsolicited incoming traffic.
> But it is... you need a bunch of new rules to pass through, limit or block a bunch of ICMPv6 messages.. there's a whole RFC just for that - https://datatracker.ietf.org/doc/html/rfc4890
With the exception of home agent, mobility and other IPv6-specific messages, many of these recommendations also hold true for IPv4. It’s just that nobody really bothers to think that deeply about it, block all ICMP and then are shocked_pikachu_face when Path MTU discovery etc don’t work.
Yeah and? How do you think IPv6 works, it’s exactly the same.
My router’s firewall’s ipv6 section help: “All outbound traffic coming from IPv6 hosts on your LAN is allowed, as well as related inbound traffic. Any other inbound traffic must be specifically allowed here.”
on openbsd pf
block outside connections from initiating connections to your hosts
block in on $external_if from any to $ip6_network
on ip4, if the world was just you would have the same rule(in ip4). however the world is not just and you usually only get one address so you have to pull some shenanigans to spoof that address across all your hosts
match out on $external_if from $internal_net to any nat-to $external_if
Really we all have a sort of Stockholm syndrome and think yes, this is normal, this is correct and being able to end to end address a host is weird and wrong.
It's not a _good_ reason, Ubiquity should've included IPv6 support from the start with the price they're asking, but it's a reason.
I remember a coworker telling me about a TV that would request and accept a DHCPv6 address and then fail hard getting to the internet. Wifi router firmware likewise messes things up, etc...
It frankly just wasn't worth the hassle. Mobile networks that can control and enforce the full stack have been able to make it work. My guess is home/wifi environments will be IPv4/NAT until the end of time, frankly.
Writing the addresses in hex is because doing so is easier. It lines up with the binary better which makes subnetting easier, and do you really want to deal with addresses that look like "32.1.13.184.133.163.0.0.0.0.138.46.3.112.115.52"?
I was wrong and the issues were elsewhere, but it remained disabled on the router for a long time.
I’m generally a person who resists change and I can’t tangibly see the benefits of ipv6; until I realised that “port-forwarding” is an exclusively NAT problem and it’s much easier with ipv6 to just natively open a port on the firewall if I want.
It also lets me wait until other people (hopefully) build better privacy systems.
I've noticed several websites where IPv6 has lower latency than IPv4. The ease of accessing different VMs on cloud providers that will hand out a single IPv4 address, though alternatives like Betternet/Tailscale/Tor will also work around that problem.
I've tested with IPv6 on and off on several machines over the course of months. Google's search results become wild and unpredictable on the same machines soon after switching to IPv4.
My theory is that they rely on that IPv6 address to know exactly who they are providing results to and thus selling to.
If that theory didn't hold water, there would be exactly zero difference in search results after switching to all IPv4.
I've noticed that many IPv6 address blocks have more up to date location information from parties like Maxmind.
No source device obfuscation equals device tracking
In fact, macOS is so aggressive about using temporary addresses that I had to turn off SLAAC in order to be able to ssh back into my desktop.
IPv6-only CVEs (these have existed!)
Maybe it's me, maybe I have a bad config or bad hardware, but it just doesn't work for me.
Next time I try I'll use those sites and see what they say.
I run a personal tech website containing OSS projects, and have been supporting v6 for more than a decade. Currently seeing few hundred unique visitors per day.
IPv6 has been steady at ~15% of all inbound requests for the past 5 years, with zero signs of increase.
Both my own fixed adsl service and a different mobile carrier do not offer v6, so I have to jump through hoops to verify my server setup.
I think you can also take advantage of router advertisement or client solicitations somehow but I’m not familiar with the details (i.e. passive listening on multicast vis a vis broadcast)
https://help.sonic.com/hc/en-us/articles/115000419168-Fiber-...
Gigabit fiber from them explicitly says it isn't supported.
https://superuser.com/questions/1055584/how-does-ipv6-affect...
- Smaller headers
- Simpler routing
ON the other hand, theoretically IPv4 header length could vary depending on the presence of IP options, so you'd need to compute the offset to continue processing the packet.
(I say theoretically because conventional wisdom is that IP options are unreliable in the face of middleboxes, so they're mostly unused. But compliant IPv4 processors have to calculate the header size anyways...)
privacy nightmare
You're fear of IP tracking is outdated.
Yes, which is why I don't use IPv6.
https://blog.apnic.net/2022/01/31/defeating-ipv6-prefix-rota...
hope you dont have any tweets that offend your goverment lol
NAT is not a privacy or security feature. You can treat it like it is, but only at your own risk.
https://www.anvilsecure.com/blog/dhcp-games-with-smart-route...
https://datatracker.ietf.org/doc/html/rfc2993#page-22
https://threatpost.com/remote-attackers-internal-network-dev...
1:many NAT does.
1:many NAT requires an affirmative choice on where to route incoming packets that aren't part of an existing stream.
In adaptation to that, most attacks are malware spread by email, or attack browser vulnerabilities, or attack services running on network devices, especially remote management systems.
How could a device have a hardware-embedded IP address, if the hardware vendor doesn't know which ISP you will use?
By putting the hardware part in the second half, and the ISP part in the first half.
You know, like how it works by default.
I agree with what you said. It's just interesting how it illustrates how different things are now.
I remember a bunch of people being horrified by the idea of 1 device per IP. I think it's more a matter of who you were around then the group changing their mind, but maybe that happened too.
The fact that peer-to-peer applications can finally start working is an unquestionable advantage.
I guess I will have to set up a VPN, which internally uses IPv4 addressing?
Is it because a large portion of the traffic is done through corporate enterprise networks and proxies? Enterprises are the ones slowing down the adoption of ipv6.
This still doesn't explain why it's six though, although I can think of four simultaneous IPv6 addresses - transient and persistent GUAs (which are accessible to the internet) a ULA (equivalent to IPv4 private address but which is rare in practice) and a link-local address (for communication to the router).
Ipv4 address just works without the hitch
Perhaps they could've required every IPv6 address to use ".." exactly once for zero compression. Then you'd have "..2001.db8.1.2.3.4.beef.de", "2001.db8..beef.de", etc.
I guess the nullary ".." could go in the middle, but that would enshrine the /64 boundary into the addressing scheme, which seems like a leaky abstraction.
Want to start a new service and require global connectivity, good luck winning IPv4 auctions because AWS is buying up all address space, even if they are not using it.
The world isn't going to end just because v4 addresses are scarce (...and I don't think anybody was arguing that it would...), but that doesn't make the problems any less real, expensive or damaging.
The security extension thing seems a bit wack. I'd still like all my traffic to originate from a single source and be tagged with that address only. This possible?
A proper firewall without NAT is generally fine, especially in combination with privacy extensions (which likely overall give better privacy than IPv4+NAT), but if you want to completely conceal the network layout behind your router, go nuts with NAT, no problem.
Give my IPv4 and NAT or turn the Internet off.
(Of course v6 adds more than just one octet, since one additional octet wouldn't be enough even for the current size of the Internet, let alone for future growth. It would be really stupid to go through all this effort, only to have to turn around and do it all again immediately afterwards because you forgot to add enough the first time around.)
It's not though. It goes through a router which has a firewall, which is the exact feature you're presumably wanting from a NAT but without any of the annoying downsides.
40% adoption after 25 years? Really highlights how terrible IPv6 is in terms of backwards compatibility.
What we needed was an internet protocol with the benefits of IPv6 that runs as an extension to the IPv4 stack.
The current approach to duplicate everything into IPv6 is wasteful and time consuming, proven by the extremely slow adoption rate.
So you will always end up in this situation where people just won't bother implementing the IPv4-replacement and you cannot simply switch to it.
Like I could envision a simple router-like device doing this.
But the problem is that the inverse is not possible. How would a IPv4-only box connect to a non-IPv4 box? How do you encode more than 32-bit of information into the 32-bit destination address field of the IPv4 packet?
My understanding is that the reason it's a new version is so existing IPv4 infrastructure would not need to be changed. This "ships in the night" approach has pros and cons, of course, but I'm personally happy to give folks who thought about this problem for many years the benefit of the doubt.
> The current approach to duplicate everything into IPv6 is wasteful and time consuming, proven by the extremely slow adoption rate.
The beauty of the IPv6 approach is that it doesn’t matter how long it takes.
I don't understand how it's "wasteful". Is it wasteful to support 3 versions of HTTP?
"HTTP/3 uses QUIC instead of TCP for the underlying transport protocol. Like HTTP/2, it does not obsolesce previous major versions of the protocol." - https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol