Usually in my day job I have to work with these tools, and currently have a stack of 5, that mostly have incomplete documentation, zero explanation of how they actually solve the problem and nearly zero ability to debug (e.g. what value have kubernetes logs and events. Usually when you have a problem is when you have no kubernetes logs yet/anymore and the events only tell you what you already know). Now I'll probably need to learn another one, considering these three options for storage.
At the weekend, where I'm mostly trying to relax and have physically only 2/7th of the time to alot I learn the basics behind containerization, e.g. namespaces, cgroups, virtual network adapters, iptables. And I feel in this small time slot I make a lot more progress at getting to that end-to-end solution that people actually need.
The example I'm using is wordpress+mysql. It's a simple thing that covers >70% of what anybody wants to deploy. And on-premise with kubernetes+docker it's still not possible without hacks (e.g. for volumeclaims and logging), after, I don't know, 4 years of Kubernetes? I bet in spending 2 years worth of weekends any normal person can come up with something better.
---
Re missing features, examples for Kubernetes:
A) Why does Kubernetes not solve the networking part? If I have a cluster and containers that may run in different places, of course the tooling I use to maintain that cluster needs to ensure that containers can talk to each other. There can be an abstract API and the option for other people to write plugins, but the core needs to come with one solution, that mostly works and is debuggable when not.
B) CrashLoopBackoff. Why did no Kubernetes developer get the idea that this state may require any kind of log/debugging?
C) Why does kubernetes assign random ports to services and not provide a simple way to retrieve them? Of course I can get them after I've learned the JSON api. But usually that is not considered a solution but hacking. I really don't care what the service is running on, I just want to use it.
D) Why do I need to manually say how many containers should run for each service? There are very distinct options a user may need. E.g., most services should run 1 instance and replace it if it dies. If it needs reliability I want to define how reliable it should be and accordingly the cluster should decide if it needs 2, 3 or 5 instances. And lastly I want to run stuff on all my nodes. Each node with a container. That is not even possible afaik.
E) Most kubernetes tools are not working well with environment proxies. For instance kube-proxy shell tool will completely bug out. But surprise, clusters make a lot of sense in enterprise environments, and enterprise environments have proxies. It's also not a hard setup for testing. A raspberry pie can be your home networks proxy.
F) on premise storage solutions, considering that a restarted container may not run on the same host.
G) Since not much is really running right now we didn't run into that problem yet. But it's totally possible that the whole cluster runs out of resources. I haven't seen any piece of info about the general cluster status and when I need to increase or replace infrastructure.
Honestly if I don't have all these things solved, am I really better of using Kubernetes or writing my own scripts? I think it currently is about equal in effort. And if that's the case my own solution has the huge advantage of being under my control and allowing me to learn a lot of new things.