393 karma · joined October 23, 2017
All the services have their own keyspace in Cassandra, there is near-zero cross service data access
We don't allow read or list of secrets by any human, although of course that's not a perfect control.
I don't think there is anything with the same guarantees as the Task ID, sadly.
Yep, so we have a different system for securely generated random keys. If its a Twilio API key, it realistically has to pass through a dev laptop, and its not that much of a big deal. If its eg an RSA key, we will generate it on an airgapped laptop, encrypt to a public key, and then we have a Vault plugin that decrypts, and writes it into Vault. So the unencrypted data is never anywhere but the airgapped laptop or Vault.
We also try to generate keys inside of Vault where possible, and we generate a lot of certificates this way.
We try to make the posts accessible to our customers as well as engineers :)
On a more general note, its taken us a long time to get comfortable with using Vault for increasingly critical things, and we are now at the point where it being down is extremely critical. But there are several components like this, and we are able to tolerate node failure, so its somewhat acceptable.