Hacker News Reader

Comment by gtaylor | Hacker News Reader

We use Kubernetes, which includes its own secrets API:

http://kubernetes.io/v1.1/docs/user-guide/secrets.html

I can't remember which issue this was on, but it seemed like there was some discussion on their GitHub project about making pluggable secrets backends (HashiCorp's Vault was mentioned).

Kubernetes' secrets API is still very basic, but I think the fundamental concept is very sound and has a great foundation to continue building on.

View on HN

Docker's commercial offering DUCP (Docker Universal Control Plane) offers this feature as well. Out in the wild, you can find Docker volume drivers for Keywhiz etc [1] that makes secrets available as files mounted to a container. I think Kubernetes does this, too.

If you are running on cloud, you would probably want your cloud provider to give you service secrets and rotate them somehow. AWS/Google Compute metadata service or Azure Key Vault are capable of doing this but I don't think they entirely map the microservices world because ACLs are set on the VM instances, not microservices specifically.

[1] https://github.com/calavera/docker-volume-keywhiz

I want to chime in with my experience.

Our service is built on top of docker deployed on CoreOS with fleet and etcd. Most of our secrets & runtime configuration was stored in etcd, which was our attempt to store the config in the environment (http://12factor.net/config).

With Kubernetes, life is much simpler. Gone are the silly fleet configuration files, and the bootstrap scripts I used to configure etcd. Moreover, Kubernetes' secrets volume means I can have my configuration and secrets easily plugged in.

There are definitely other great solutions out there, but I'm sold on Kubernetes.

> I can't remember which issue this was on, but it seemed like there was some discussion on their GitHub project about making pluggable secrets backends (HashiCorp's Vault was mentioned).

https://github.com/kubernetes/kubernetes/issues/10439#issuec...

Are the k8s Secrets still stored in plaintext in the etcd datastore? Seems that this feature is a bit half-baked right now -- though I'd love to me mistaken on this point. The k8s docs mention shredding your apiserver hard drives once you're done with them; that's hardly feasible in a cloud environment.

Also on access control, any process with root on any node in your cluster can get access to all your secrets (since the kubelet needs to be able to do so). There are no user access controls either; any cluster admin can dump all the secrets.

This stuff is clearly documented, so it's not an indictment on k8s; I just get the feeling that the feature isn't really ready for production use yet.

afaik the only part of kubernetes accessing etcd is the master. Nodes don't need and can't access etcd directly.

That still leaves the secret in plain view on the nodes that run the pod that needs the service. It would be great to be able to umount the secret when not needed anymore.

Correct, the etcd instance is only accessed by the master, which uses etcd to back the apiserver. But any root process on the nodes can access the secrets through the apiserver (there's no access control at this point).

Reply on news.ycombinator.com