It's an attempt to replicate the old model of "hard exterior, gooey interior" model of corporate networks.
I would very much prefer if K8s used public routable IPv6 for traffic delivery, and then simply provided an authenticated overlay on top of it.
> I would very much prefer if K8s used public routable IPv6 for traffic delivery
shudder... nothing could feel more wrong to me than public routable IPv6, yuck.
Public IPs also definitely don't need to be accessible from the wide Internet. Border firewalls are still a thing.
Btw, overlay is not the only option to do CNI - Calico, Cilium and few others can do it via l3 by integrating with your equipment. Even possible in cloud but has serious scale limitations...
IPv4 is needed only for the external IPv4 clients, and for the server code to reach any external resources that are IPv4-only. The clients are simply going to connect via the border load balancers, just as usual.
For the external IPv4-only resources, you'll need to use DNS64. But this is not any different from the status quo. Regular K8s nodes can only reach external resources through NAT anyway.
I'm actually trialing this infrastructure for my current company. We got an IPv6 assignment from ARIN, so we can use consistent blocks in US West and US East locations. We don't use K8s, though. AWS ECS works pretty great for us right now.
> Btw, overlay is not the only option to do CNI - Calico, Cilium and few others can do it via l3 by integrating with your equipment. Even possible in cloud but has serious scale limitations...
It's still an overlay network, just in hardware.
It really isn't, at least not in commonly understood sense. See [0] for example - you can use this with dual-stack and route everything natively even with ipv4 using rfc1918 cidrs. No ipip/gre/vxlan tunneling required. Does require setting up BGP peering on your routers.
[0] - https://cloudnativelabs.github.io/post/2017-05-22-kube-pod-n...
I don’t agree with your approach (curious as to why you would want this) but I believe it’s possible.