Hacker News Reader

Kubernetes is a complex project with a large codebase. Writing it in bash would be a nightmare to maintain.

Go makes sense as an implementation language for a system like this for many reasons:

- it doesn't require a VM (goodbye Java) - it is a memory safe language (goodbye c++) - it has strong types to improve reliability - it does a good job of handling complex, multi-module code efficiently (these two say goodbye to python)

So of the Google friendly languages you end up with Go.

And I don't know if it was planned, but Go has turned out to be a boon for contribution. A surprising number of devops engineers are willing to dip their toes in Go waters and I highly doubt they would've done so with Erlang.

I think you'd end up going the chef route, writing the control plane in Erlang and expecting users to interact in Ruby or something like it.

Also this whole conversation seems a bit off because Kubernetes is a multi-service architecture with many components. You can absolutely write operators in other languages since Kubernetes exposes an API.

It's kind of the whole point to be able to take an existing application and run it in k8s rather than a regular VM with minimal changes. Expecting developers to rewrite everything in Erlang is nuts.

> And I don't know if it was planned, but Go has turned out to be a boon for contribution. A surprising number of devops engineers are willing to dip their toes in Go waters and I highly doubt they would've done so with Erlang.

> I think you'd end up going the chef route, writing the control plane in Erlang and expecting users to interact in Ruby or something like it.

DevOps engineers have learned to be comfortable with Ruby (Chef) and Python (Ansible) so this certainly makes sense.

> Expecting developers to rewrite everything in Erlang is nuts.

...But I never said that?

You are correct on your points and I don't disagree. Golang is certainly a much better choice than Bash indeed. You are also correct on developer willingness to work with Golang.

I am aware that K8s and Erlang/OTP are apples to oranges comparison; they serve different needs. Whereas Erlang's runtime (the BEAM VM) can give you fault-tolerance, K8s tries to do roughly the same on a higher level - it tries to give you throwaway containers that can be switched off and on at any time.

As I mentioned in my parent comment, I admire their work.

What my point was that if you squint your eyes hard, it kind of looks like K8s wants to invent Erlang/OTP for infrastructure (as opposed to Erlang/OTP which gives its guarantees per node).

> which gives its guarantees per node

There is no such thing, because Nodes (and even entire datacenters) can fail.

In other words, node failure is an infrastructure problem that is best NOT handled by your bespoke application code. Replacing failed nodes should NOT be custom code in your app, that way lies madness.

> kind of looks like K8s wants to invent Erlang/OTP for infrastructure

You can say "K8s and Erlang implement similar ideas implemented at different levels." But you can't pretend one is a substitute for the other, nor that "Erlang has done everything that K8s can do."

E.g. Writing in Erlang doesn't magically get you:

- deploy/upgrade of the Erlang runtime, including rollback + multiple versions co-existing, including any compiled foreign code (which is required in the article). - ability to log, monitor, probe and reroute the connections between services -- In a standard way such that the application doesn't have to be modified.("service mesh") - Ability for an entire ecosystem of tools to inspect the versions of your application services that are deployed, because it's exposed as an API. - A standard ecosystem of plug-ins for operators (autoscale, autoscale to EC2 spot instances, capacity planning, "best practices" of running a MySQL cluster, etc.) None of these should ever be mixed with the application. (Unless you are Kelsey Hightower: https://github.com/kelseyhightower/hello-universe )

> There is no such thing, because Nodes (and even entire datacenters) can fail.

Gross over-simplification that I'd also call a strawman. In the face of lack of electricity, of course no computer language matters at all. What's your point?

> it doesn't require a VM (goodbye Java) -

Neither does Java, only devs clueless about Java world aren't aware of AOT compilers to native code.

Also sorry to spoil your fun, Kubernetes was originally written in Java, it was later rewritten in Go as other team took over.

Apparently a prototype was written in Java. Borg was written in c++. AFAIK Kubernetes was always a Go project, but maybe I'm wrong. Why do you think Go was chosen?

Your comment about my experience with Java was needlessly provocative so I'm going to ignore it.

It was mentioned at the FOSDEM 2019 talk about K8S.

Nowhere in my commented I addressed the Java remark to your specific person.

It was targeted to all of those that bash Java without actually knowing how the rich the eco-system actually is.

You called me a "clueless dev".

No I did not, nowhere on my sentence is the direct mention of your person.

On my previous comment I explicitly mention it was targeted to developers that bash Java, without having any clue what they are bashing about, cargo cult if you wish.

So either you feel offended, because you are indeed attacking Java without having any clue about the Java eco-system, or are deciding to play victim here, when I never mentioned you directly.

This is a public forum, so I leave to everyone else and HN moderators to judge my comments, and won't reply to anything else on this thread.

Kubernetes is based on Google Borg / Omega not OTP very different things. As for your queue / DB question there is something called transactions for that kind of problem, I hope you don't use OTP features replacing transactions or you're looking for very serious issues.

OTP doesn't do 5% of what Kubernetes is able to provide ( cpu / io / memory quota, live / readiness probes, rolling deployments / canary / blue green.

I knew somebody will get my random examples and turn them against me. :D Serves me right for not researching several hours to find the perfect example, I guess!

I am well aware of the differences in scope between K8s and Erlang/OTP. My point was that they kind of try to do the same only on different levels.

And I am still not sold on the idea of throwaway containers. That works well if you have a hyper-network of microservices that are able to discover each other and self-heal a bigger graph of services but if you have any sort of a more classic coherent whole app... then not really. Throwing containers away and rebooting new copies isn't something that looks viable for many projects.

But hell, who knows. K8s team and their audience are really dedicated. They might change every single tool of the trade just to make K8s work well. I was mostly saying that I don't feel it brings something really radical to the table.

Let it crash is kind of overstated for Erlang too. Sure, a supervised process will restart, but you'll lose the message queue. For a gen_server that's mostly stateless (or that's mostly servicing state in mnesia or similar), it ends up being less disruptive to catch in handle_call around your real work, so a crash only throws away the one request.

Caveat: in the environment I work, we don't do real gen_server:call, because all of the included monitor/demonitor calls are too expensive for us. The trade-off is we only have timeouts when the server goes away during the request. If you had the monitor letting you know the request crashed the server, you could presumably do a smartish retry -- but it's still nicer to only need to do that for the crashing request, not the others that would fail simply because the server died.

Depends on the scenario of course, but when working with Elixir I tend not to use GenServers much as well. I mostly handle problems with idempotent and at-least-once-executed tasks. Works quite well 99% of the time.

This is why HashiCorp's Consul/Nomad/Vault/Terraform stack has strong support - it is way easier than setting up K8s.

Better yet: a whole pile of microservices, preferably in node.js that tries to re-invent supervision trees and of course on Kubernetes.

If you want to re-invent the whole Erlang eco-system you should just switch and call it a day, the chances that a company that needs to get a job done will be able to pull this off successfully as a side project are nil.