Kubernetes State Checker | Hacker News Reader

Kubernetes State Checker

github.com

46 pointsceohockey6012 comments

> You might ask, with Kubernetes, don’t you declare the state you want and Kubernetes makes that happen

Am I using Kubernetes wrong, or is this not really the case?

You say "make this yaml file happen", but any given yaml file is a small fraction of your deployment. It doesn't make all the other yaml files happen. It doesn't make the yaml files you no longer want happening, stop happening. It doesn't even tell you "these things were created by 'old' yaml files" and should be garbage collected (since it doesn't seem to have a sense of old yaml files).

It's lifted the scope of commands from one `cp` or `apt-get install` at a time, to one yaml file at a time. But it still feels like I'm managing the state of servers by running a sequence of commands, instead of just declaring what it should be?

Theoretically, in k8s, you should deploy once, and then things work. You may be missing parts of the puzzle such as resources like jobs and deployments or tooling like Kustomize or skaffold.

> It doesn't make all the other yaml files happen. It doesn't make the yaml files you no longer want happening, stop happening. It doesn't even tell you "these things were created by 'old' yaml files" and should be garbage collected (since it doesn't seem to have a sense of old yaml files).

This is definitely one piece of Kubernetes that is getting a lot of attention recently. The three tools I've been paying attention to are Argo CD[0], Flux[1], and Config Sync[2].

All of these allow you to point your repository to a cluster and sync resources from the repo to the cluster, including deletes.

[0] https://argoproj.github.io/argo-cd/

[1] https://github.com/fluxcd/flux

[2] https://cloud.google.com/kubernetes-engine/docs/add-on/confi...

Disclaimer: I work at GCP, but not on the GKE team. Opinions are my own.

Don't forget Terraform. It's been doing this for me in Kubernetes for a few years now.

Or helm. (Or helm via terraform.)

As someone within GCP, do you know / can you find out what the story is with Application Delivery (https://cloud.google.com/kubernetes-engine/docs/concepts/add...) ? It looks like a promising idea, but it’s half-baked† and showing no signs of progressing to becoming fully-baked. Is it still being actively worked on? I’d love to use it — compared to Flux/Argo, App Delivery has semantics closest to what I want — but only if there’s some promise of it reaching GA and receiving some more polishing.

† I could list off the reasons it’s half-baked, but you’re not the target of that ire, so I’ll leave off with it :)

There was one blog post about App Delivery (https://cloud.google.com/blog/products/containers-kubernetes...) in early 2020, then nothing. There’s no official site, or reference, for it, or the components within it.

The Application Delivery client appctl(1), despite being a standalone static Go binary, isn’t available separately — the only place to get it is from the “pkg” component of gcloud(1). Nor is there any way to deploy the cluster-side components other than through the GKE dashboard/gcloud(1) CLI. So there’s no good way to deploy+test Application Delivery in your own (e.g. minikube) cluster, to test ‘doing GitOps with Application Delivery’ in a sandbox.

Nor is there any promise of App Delivery becoming tooling that’s portable to other Kubernetes hosts. Nothing about any of this is open-source or even source-available — despite App Delivery advertising itself as a “best practices” approach to GitOps, something intended to show other vendors how this kind of thing should done. (How are they going to learn GitOps from a black box?)

—————

Application Delivery seems to be based on some (Google-internal? Unpublished, at least) software called KALM (which, I assume, translates to “Kubernetes Application Lifecycle Management” or some such.)

Installing Application Delivery into a cluster installs “kalm-controller”; and appctl(1) seems to be at-least-in-part a fork or wrapper for a “kalmcli” client. (I learned this by digging through the debugging-symbol table of appctl; it’s not documented anywhere.)

It seems that, because of this separation-of-concerns, KALM itself isn’t supposed to be GKE-specific, whereas Application Delivery is.

Well, if an open Application Delivery isn’t in the pipeline, is an open KALM? Or was it ever (and did it stall—perhaps because the lead on it left the project?)

(I notice that KALM also has something to do with how Google Cloud Marketplace “Click to Deploy” k8s applications are packaged, which at least leads me to believe that unlike Application Delivery itself, KALM is probably still a living project.)

I like ArgoCD a lot, it gives good visibility into the reconciliation and deploy process. If you have a human approving a release, it’s particularly useful.

I do think k8s would be improved with a “PUT /namespace/myapp“ command that specifies all resources in the namespace. Currently adding resources to a ns is easy, but you need smarts like Argo to make sure unwanted resources get cleaned up. A simple example would be if you rename a Deployment, the old one won’t get deleted when you ‘kubectl apply’ the new one.

A lot of simple pipelines could be supported with a simple stateless PUT that defines the totality of the desired state for a namespace.

(Also nested namespaces might be more relevant in this model too).

ArgoCD is a delight.

In Kubernetes there is the "Desired State", which is the sum of all resources (your yaml files) and the "Actual State".

Whenever a change is requested on the "Desired State", Kubernetes will validate it (using RBAC, Admission Webhooks, ...) before persisting it (in etcd in most cases). Then after the persistance is done, it will notify the operators (watching the resources) that are in charge of performing the side effects (run that docker image, allocate that storage volume, ...), moving the "Actual State" closer to your "Desired State".

Creating a new Custom Resource Definition and an operator helps you hide the complexity. For example:

  - the (builtin) Deployment operator creates a ReplicaSet
  - the (builtin) ReplicaSet operator creates Pods
  - the (builtin) Pod operator runs Docker images and updates the Pods status

Nothing prevents you from making an operator that creates Deployment/Services/Ingress for you.

This is even simplified with Helm (to template your resources) and the HelmOperator[1].

[1] - https://github.com/fluxcd/helm-operator

For the port checking problem, aren’t named ports a potential best practice/solution?

    export GOPATH=$GOPATH:$PWD

Wasn't the need to do that removed by the use of go.mod? Roughly the same question about the 8 or so manual (and unpinned) `go get` commands

Pedantically, the command should also be

    export GOPATH=$GOPATH${GOPATH:+:}$PWD

otherwise one will end up with a GOPATH of ":/the/dir"

Can write helm tests which seems like the same thing as this

Reply on news.ycombinator.com