I am curious to see how they recommend handling storage on bare metal k8s.
- Application servers are stateless.
- All state lives in stateful databases.
- Those database handle high availability at the application layer by replication data, so you can use local storage and deploy them using StatefulSets. Has the extra benefit of avoiding the network storage latency penalty.
We're dealing with that exact issue right now. Our K8s cluster is not playing nice with a NetApp NAS. We are exploring other options such as local iscsi storage volumes ($$$) or an external state store db (slow).
[1] - https://rook.io/
I heard from someone at a large company, though, that it's not getting a lot of love from Red Hat nowadays, even if you're a large paying customer. Now I'm curious.
That announcement gave me the impression the project is making active progress.
My team is still very interested in Ceph and Rook, for the record.
These days I usually recommend that any data an app wants to persist be written to an object store or database.
And that any app that needs to write critical data to a disk should be provisioned with config management on bare metal (or pods statically assigned to machines with attached storage if K8s is a must).
This is mostly due to how tough the problem of storage is on K8s, and how much is at risk when it goes south.