What is the simplest/easiest personal project where using Kubernetes might be justified?
I am a junior software engineer trying to figure out how to contextualize all of these container/container management systems.
1. at least six independent twelve-factor-app services with their own datastores and a need for high availability across all of them and a near-complete understanding of the high-availability interactions between instances of your services
2. an inability to predict ahead of time where your system's hot spots are, necessitating rapid scaling of different parts of the application
3. a willingness to overspend on capacity to be able to respond to scaling events or deploys in seconds rather than minutes
4. an code-focused ops team (as opposed to a mouse-driven ops team) with extremely strong diagnostic skills and the bandwidth to babysit a service with a potential pain-in-the-ass ceiling around that of a Cassandra cluster
Without #1, you don't have enough variation in systems to benefit; just stick a monolithic application in an autoscaling group. (Most people should do this.) Without #2, you can lean into the hot spots of your application by scaling them horizontally--bear in mind that you'll be paying for capacity you don't use with k8s in order to get that environmental reactivity, so you could just spend that on making your hot spots faster. Without #3...well, that one's pretty obvious when you look at things like EC2 instances, which are more easily partitioned, can be spun up in smaller/cheaper groupings, and their primary downside is that it takes longer than deploying a container. And without #4, you're gonna go off the cliff.
Reasonable people can nibble at the edges. But to answer the thrust of your question: it's probably never reasonable to design a personal project around k8s unless the point of the project is to be done on k8s.
Here it is running fine... running fine.. running fine... aaaaand there's a compaction-and-gc cycle of death and fire and lost data and tears. Thank you for this terrible memory.
Everything is easy when it has nothing riding on it. When it isn't is where the value of a tool comes into focus.
I've been recently asked why I'm extremely restrictive and careful with our primary production cluster. Well, we got 20k+ full time employees of our customers depending on this system for their everyday work. An hour of downtime of this thing will cost our end customers 20k man-hours of work done in a worse way.
We're not touching the tooling this system sits on without good reason and a lot of testing. And even then I'll be bloody scared. Sorry modern world, but in this case, I'll be wearing my hard ops hat.
- High availability services (more than one copy running at once).
- Service discovery (services talking to each other in a resilient way).
- Ability to automate operational tasks.
- Rolling deployments of services.
Very few personal projects will tick those boxes -- by the time they do, they've usually evolved into a full-on "real project".
Doesn't mean you can't mess around with Kubernetes for fun and learning, of course. But from a purely practical perspective, it's overkill unless you have all of those requirements above. (If you just have one or two of those needs, there is usually a simpler tool to fulfill it with less overhead).
For a simple deploy you probably don't need Kubernetes or even containers.
If you are running containers, you'll need a mechanism for running them. And maybe at some point you want something to recreate them when they die or become unhealthy. Maybe you want to run multiple containers, and you want to do rolling deploys of them. Maybe you want to run them on multiple hosts and network them together. Maybe you want to be able to attach a persistent disk to some of them, or interface with some secrets management software. And maybe you want a single, well-supported API for doing all of the above.
There's a lot more that can be said about Kubernetes; it offers a lot out of the box as well as an API for extending it when you need behavior it doesn't provide.