You can use Vault(https://github.com/Boostport/kubernetes-vault) or extend Kubernetes easily for your specific secret as a replacement if you need extra security.
You can use Vault(https://github.com/Boostport/kubernetes-vault) or extend Kubernetes easily for your specific secret as a replacement if you need extra security.
Then you need a mutual authenticated connection from the api server to etcd + encrypted etcd hard drives + networking policies that only the api-servers are able to communicate with etcd and your secrets are pretty safely stored.
This seems like a fairly difficult problem to solve with meaningful security confidence. The only easy way I can see this working is if you pre-define groupings of kubelets and specify which pods may run on which kubelets, and enforce secret access the same way. Without this kind of hard separation, any kubelet is a candidate for running any pod and needing any pod's secrets at a moment's notice.
Furthermore, this sort of separation would require some kind of PKI; you could not just trust any given kubelet to accurately claim which groups a member of. You'd need to provision the kubelet groups with a secret that proves their membership in the group, and we're back to the secret distribution problem again.
And while it's true that a given kubelet may not be running a certain pod at the moment (and thus doesn't need the secret), that does not seem to be a hard security boundary. An attacker who controls the kubelet can manipulate which pods run on it, such as by terminating pods until the desired one is scheduled on it, or falsely advertising a huge amount of available resources to entice the scheduler to run pods on it, and so on. Ultimately if a kubelet is a candidate for running a pod, then it is simply a matter of coincidence whether it possesses the pod's secrets at a given moment or not.
That said, limiting kublets to have access only to secrets required by active pods will make things harder for an attacker, and so will provide value. But we should also evaluate the priority of that work in context of how hard it will be for an attacker to defeat that same restriction (e.g., advertise 1 petabyte of free RAM and disk, and 1000 idle CPU cores).
Coincidentally, I'm working on a project that uses Kubernetes and it has a very locked down pod placement policy, so the attack you described would be significantly scoped down. But I don't think the same is true of most Kubernetes deployments.
Securing nodes, preventing container escape, subdividing role access, constraining placement, limiting master surface area, and end to end audits have been the initial focus. Until those are in place, node secret access was less critical.
It is something that several folks are interested in working on soon.
Everything takes time to design and implement, and human power to work on the hard/not-fun things is pretty hard to come by in such hugely active projects...
Looking at your profile/comments you seem to be openly hostile towards Kubernetes. On that note: this is quite representative of the mentality I encountered while working there. Open Source is hard and bashing other projects is not something I consider to be fair game (regardless of whether the other side is adopting that stance or not).
Meanwhile, I admire Hashicorp's attitude of focusing primarily on improving their product without taking part in this silly "orchestration war".
(and thanks to cpuguy83 for having a more sensible view of what it's like to work on Open Source projects)