Hacker News Reader

Comment by AtlasBarfed | Hacker News Reader

So you don't run any databases in those thousands of clusters?

To your point, and I have not used k8s I just started to research it when my former company was thinking about shoehorning cassandra into k8s...

But there was dogma around not allowing access to VM command exec via kubectl, while I basically needed it in the basic mode for certain one-off diagnosis needs and nodetool stuff...

And yes, some of the floated stuff was "use sidecars" which also seemed to architect complexity for dogma's sake.

View on HN

> So you don't run any databases in those thousands of clusters?

We do, but not of the SQL variety (that I am aware of). We have persistent key-value and document store databases hosted in these clusters. SQL databases are off-loaded to managed offering's in the cloud. Admittedly, this does simplify a lot of problems for us.

How much data? I keep hearing k8s isn't usable becasue sometimes there is to much data and it can't be moved around.

In the managed k8s space, the data is on a PVC in the same availability zone as the node it is being mounted on. If the node dies, the volume is just mounted on to a new node in the same zone. There is no data movement required.

While I've not played with k8, I did run stuff in Google's Borg for a very long while, and that has a similar architecture. My team was petabyte scale and we were far from the team with the largest footprint. So it is clearly possible to handle large scale data in this type of architecture.

The simplest approach I’m aware of is to create the k8s cluster and databases in the same datacenter / availability zone.

postgresql operators are pretty nice, so it makes sense to run stateful stuff on k8s (ie. for CI, testing, staging, dev, etc.. and probably even for prod if there's a need to orchestrate shards)

> exec

kubectl exec is good, and it's possible to audit access (ie. get kubectl exec events with arguments logged)

and I guess and admissions webhook can filter the allowed commands

but IMHO it's shouldn't be necessary, the bastion host where the "kubectl exec" is run from should be accessible only through an SSH session recorder

Reply on news.ycombinator.com