Hacker News Reader

Comment by bradleyjg | Hacker News Reader

Maybe I'm missing something, but how do you get the client certificate private key on the e.g. new webserver which allows it to connect and retrieve its server certificate private key? Isn't that the same key distribution problem, just one step removed?

View on HN

Each client would have its own client key in this case, compartmentalizing access. The loss of a single client key would not expose your entire infrastructure.

I had been planning to do this with keys stored in S3 and IAM roles to retrieve the keys at instance boot time, stored in a ramdisk, but this saves all of that trouble.

Instance profiles (credentials) are accessible to all users for the life of the instance. Until AWS provides an ability to lock down portions of instance metadata and/or a way to delete early in the boot process, it's much safer to bake your bootstrap key directly into the AMI as root (or specific user) visible only.

Good point. I've been using IAM roles to bake the AMIs with Packer, where the key is fetched during the bake process; perhaps I'll investigate sneaker if there's time tonight.

Thanks for rubber duckying this with me!

Yeah, perhaps as an additional block device you can mount with whatever permissions you want.

right on, like the FUSE in question.. but securely mounting something remotely means storing a bootstrap key somewhere anyway. Turtles all the way down :)

Fundamentally you dont. Or ive never seen an actual solution in practice. Physical hardware comes down to trusting the layer one/two domain and (usually) the mac address. With virts you trust your infrastructure to pass through some blob like a pre shared key. Which again is just punting on the actual problem. In either instance you might inject manual authn/authz by requiring a human in the process.

Hypothetically the problem of boot strapping trust is solved by tpm. However thats predicated on functional hardware attestation. And thus the hypothetical bit.

https://github.com/square/certstrap

I saw a reference to that PKI package in the linked article, but it doesn't really answer the question.

A newly brought up server can generate its own private key and issue a key signing request to a certificate authority, but how does the CA server authenticate the request? You still need a shared secret (perhaps implicitly through the username / password of an administrator doing things manually).

Any way you cut it, I don't see how you avoid this problem outlined in the article: "Common practices include putting secrets in config files next to code or copying files to servers out-of-band. The former is likely to be leaked and the latter difficult to track."

I do take the point made by some of the sibling comments that this allows for better mitigation of the damage in the case of the loss of the bootstrap shared secret, and I can certainly see the value in that.

Perhaps similar to how new versions of puppet let you provide custom attributes (like a pre-shared key) as part of the request so you can do verification on your issuer and build the secret into your boot strap process. A key compromise would simply require a rotation on your issuer and your boot strap process.

You can solve this by having a bootstrapping process that issues the appropriate credentials when bringing up a new server.

And how do you trust the identity of the new server/instance during boot strapping?

You could leverage the TPM and some version of remote attestation and only permit key-requests from attested machines. Alternatively (or concurrently), you could PXE boot all devices with a parameterized shared-secret individualized for each node.

I'm getting the feeling they don't like something about etcd or raft or both. Is that accurate? I noticed certstrap was adapted from etcd-ca.

It certainly requires some bootstrapping, but it seems like once you've got the certs setup keywhiz should be able to manage updates to those and all your other keys, so you do the annoying work once with what are essentially ephemeral secrets, and then you get to use keywhiz to do all your management thereafter.

I get the impression that you're going to need a certificate creation process to go with it, probably with an in-house authority, to allow the components to talk to each other in the first place.

Reply on news.ycombinator.com