Melingo

No clue how this works.

So everyone has to earn at least below 50k?

I mean overhead cost, building rents and c level salaries?!

And?

My experience still counts for something and the example with those 700 VMS is something I didn't just saw once.

That's a real good point you mentioned: k8s ecosystem is super young.

And so so much changed in the last 4 years.

But at least for me, the 'easy to use' threshold happend somewhere like 2-3 years ago.

And Gardner for example upgrades quite well.

Rke2 is quite stable for me but rancher integration is still not perfect.

But even doing k8s by hand with Ansible was already double 3 years ago. That's what I started and I had it up and running. I switched to rke2 because I realized that this will not be sustainable/ is not worth it to do it myself on this level.

Kubernetes is a poster child on open source and transparent development.

And it pushed the needle tremendously.

I'm lost on how you can compare this even.

In all managed services all of that comes out of the box.

Ingress, lb, storage, network...

And I have my small setup running with all of it too. Took me a weekend to set it up.

Rke2, nginx ingress, classic lb in front, cert manager and everything else in argocd.

I am very curious why they had such a bad k8s experience while mine is the total opposite.

And just inventing known stuff doesn't mean they are infra or platform experts.

I grew into cloud in the same time frame as they did and I'm also a platform expert and k8s expert.

So are they automatically better in deployment and platforms because they invented ruby on rails?

Depends on the quality and hr overhead.

Oncall, new features, security etc.

No small it department I worked with, was anywhere close to the features and quality of Google and co.

And I didn't say that.

We had all of these problems with self developed automatisation.

It still was garbage.

K8s just solves those issues out if the box.

Nope Ansible is horrible in comparison to k8s.

Alone the Paradigma shift from doing things step by step vs describing what you need and than things happen on it is a game changer.

K8s is probably 100x easier than Ansible.

And Ansible also has it's bigger ecosystem like Ansible tower.

Basically your k8s control plane but in bad

Have you tried using gardener or rancher or ubuntus solution on bare metal?

They are very easy to use

It definitely does not.

The main issues we faced with over 700VMs were: outdated os, full disks, full inodes, broken hardware, missing backups or missing backup strategy, oom.

K8s health itself, fixes out of memory by restarting a pod, solves storage by shipping logs out and killing a pod in case it still runs full, has a rollout startegy, health checks and readiness probes.

It provides easy deployment mechanism out of the box, adding a domain is easy, certificates get renewed centrally and automatically.

Scaling is just a replica number and you have node Autoupgrade features build in.

K8s provides what people build manually out of the box, certified, open sourced and battle tested.

If you need compute there is nothing worse than old hardware.

It's just not worth it at all.

Ram is much slower, energy consumption is higher, core count and caches are lower.

Alone caches can hurt you a lot. I had to upgrade my CPU after I got a new DSLR because the new raw files just killed it and I'm pretty sure it was the cache sizes

K8s is very easy to keep maintained, especially because even for self hosted there are plenty of management tools around.

And it allows your teams to deploy everything they need without a admin team.

Even ingress you can define

Because they already had it running in k8s.

And k8s scales very well to very low and high numbers.

Because k8s provides battle tested Features like rollout lb etc.

And the ecosystem is great.

Certmanager, argocd kube stack.

I'm baffled tbh how they had such a difference experience with k8s than I do

And don't get me wrong: whatever works for the company but k8s experience alone is already super helpful.

A lightweight k8s stack out of the box + argocd + cert manager is like I fra steroids.

I'm so lost on so many choices this company did.

You de cloud and now use some mini tool like mrskd?

I'm running k8s on azure (small), gke (very big), rke2 on a setup with 5 nodes and k3s.

I'm totally lost why they would de k8s after investing already so much time? They should be able to work with k8s really well at this point.

Sry to say but for me it feels the company has a much bigger issue than cloud vs non cloud: no one with proper experience and strategy.