Comment by alphazard | Hacker News Reader

Comment by alphazard | Hacker News Reader

Parent Full thread

The K8s master is just a scheduling application. It can run anywhere, and doesn't depend on much (just etcd). The kublet (which runs on each node) is what manages the local resources. It has a plugin architecture, and when you include one of each necessary plugin, it gets very complicated. There are plugins for networking, containerization, storage.

If you are already running SEL4 and you want to spawn an application that is totally isolated, or even an entire sub-kernel it's not different than spawning a process on UNIX. There is no need for the containerization plugins on SEL4. Additionally the isolation for the storage and networking plugins would be much better on SEL4, and wouldn't even really require additional specialized code. A reasonable init system would be all you need to wire up isolated components that provide storage and networking.

Kubernetes is seen as this complicated and impressive piece of software, but it's only impressive given the complexity of the APIs it is built on. Providing K8s functionality on top of SEL4 would be trivial in comparison.

I understand what you're saying, and I'm a fan of SEL4. But isolation isn't one of the primary points of k8s.

Containerization is after all, as you mentioned, a plugin. As is network behavior. These are things that k8s doesn't have a strong opinion on beyond compliance with the required interface. You can switch container plugin and barely notice the difference. The job of k8s is to have control loops that manage fleets of resources.

That's why containers are called "containers". They're for shipping services around like containers on boats. Isolation, especially security isolation, isn't (or at least wasn't originally) the main idea.

You manage a fleet of machines and a fleet of apps. k8s is what orchestrates that. SEL4 is a microkernel -- it runs on a single machine. From the point of view of k8s, a single machine is disposable. From the point of view of SEL4, the machine is its whole world.

So while I see your point that SEL4 could be used on k8s nodes, it performs a very different function than k8s.

The scheduler is the least interesting thing about k8s. The extensible API common to all operating environments is the real value add.

As others mentioned containers aren’t about security either, I think you’re rather missing the whole purpose of the cloud native ecosystem here.

> Kubernetes is seen as this complicated and impressive piece of software, but it's only impressive given the complexity of the APIs it is built on.

There are other reasons it's impressive. Its API and core design is incredibly well-designed and general, something many other projects could and should learn from.

But the fact that it's impressive because of the complexity of the APIs it's built on is certainly a big part of its value. It means you can use a common declarative definition to define and deploy entire distributed systems, across large clusters, handling everything from ingress via load balancers to scaling and dynamic provisioning at the node level. It's essentially a high-level abstraction for entire data centers.

seL4 overlaps with that in a pretty minimal way. Would it be better as underlying infrastructure than the Linux kernel? Perhaps, but "providing K8s functionality on top of SEL4" would require reimplementing much of what Linux and various systems on top of it currently provide. Hardly "trivial in comparison".

You're just replacing the functionality of CRI, which is already pluggable. Rest of the kubernetes is still needed.

You have solved the isolation and some storage problems for a single node. You have not solved for scaling that to 10s, 100s, 1000s of nodes. That's where Kubernetes comes in. You made a lot of good points, but "you no longer need k8s" is not one of them.

Reply on news.ycombinator.com