Comment by theptip | Hacker News Reader

Comment by theptip | Hacker News Reader

The key thing here for my use-case is this point (which took a lot of digging to uncover when I last dove into this a year or so ago):

> Data in transit inside a physical boundary controlled by or on behalf of Google is generally authenticated but not necessarily encrypted.

Previously I believe this was not explicitly called out, but this is very important for GKE! In the default configuration, Kubernetes can arbitrarily bounce Service traffic between nodes, since the cloud LB selects a node at random, and then the Service iptables rules redirect the traffic to a node which hosts a pod that backs the Service.

So if your regulatory environment (or SLA) requires end-to-end encryption, you are not covered using GKE out of the box.

Options I've found to resolve this:

1) TLS to-the-pod

2) Using source-IP-address-preservation to ensure that the Service doesn't reroute your traffic to another node.

I'd really prefer if Google made this limitation a bit clearer in their GKE docs, since it's a major security gotcha, and took me a lot of digging to piece together. But it's definitely a big step forwards that the encryption policy is spelled out explicitly here.

Does this section on Istio apply to your use-case?

> If you want to implement mutual authentication and encryption for workloads, you can use istio auth. Specifically, for a workload in Kubernetes, Istio auth allows a cluster-level CA to generate and distribute certificates, which are then used for pod-to-pod mutual Transport Layer Security (mTLS).

https://cloud.google.com/security/encryption-in-transit/#ser...

That was our intent (though we can always be clearer). If your regulatory environment requires that level of encryption, you'll want to use something, like Istio, that provides pod-to-pod mTLS (likely via a proxy or service mesh or both).

Disclosure: I work at Google on Kubeflow

My use-case is currently quite simple (very few services) and so getting a TLS cert into each pod is easy enough. (I use the cluster CA to manually provision certs for the small number of private services that I have, and Acme certs for all of my publicly accessible services).

As I roll out more services I've definitely got my eye on Istio, since auto-provisioned mTLS certs will be very useful. It does seem a bit bleeding edge though; it's only on version 0.3.

this is a big deal for a lot of people. I was always under the impression (I'm pretty sure I had someone at GCP tell me at one point too) that basically all network communication was encrypted behind the scenes, even within your infrastructure. This is clearly not correct.

I asked questions about this exact same thing during a deep dive on EKS at reInvent a couple weeks ago; unsurprisingly the same issue exists there.

Right. Encryption in transit within a building at line rate (and more importantly low latency) is hard. To my knowledge, none of the providers do this currently, otherwise they'd be talking about it. This is us saying it's time to talk about it :).

Disclosure: I work on Google Cloud.

Military stuff uses "XOR the data with a secret keystream", which is zero latency (okay - more like 10 nanoseconds of latency), and can run arbitrarily fast.

The only disadvantage is one has to use a stream cipher, and those have fallen out of favor lately...

Is generating a cryptographically secure secret stream that fast?

If you want to, sure.

You can do it fully in parallel. Imagine 1000 crypto units each producing 100 Mbps of key-stream. All on one ASIC.

Then all you need is to interleave the stream and XOR with the data to encrypt your 100Gbps stream.

The latency isn't from the cipher; it's from touching the data an additional time. A NIC with inline encryption adds negligible latency but Google probably doesn't have those.

If anyone were to have those, it would be google.

Since they design their own NIC, Google certainly has the capability to include whatever features they want. Yet Andromeda appears to be implemented in software.

The problem is that Google (and all other large cloud providers) use an SDN, and so you can't just encrypt traffic at the NIC level -- you need to just encrypt the encapsulated overlay traffic, while leaving the underlay unencrypted (so that intermediate nodes can route the overlay network traffic).

There are a number of technologies that support this kind of thing at the kernel level, but 1) layering these onto an existing SDN is not trivial, and 2) the extra encoding/decoding will absolutely have a performance impact if you're trying to do it at line rate on a 10Gb NIC, e.g. see https://www.weave.works/blog/weave-net-performance-fast-data....

Reply on news.ycombinator.com