Hacker News Reader

Comment by verdverm | Hacker News Reader

If they are reluctant and only do it because they have to, are they really the right vendor for managed k8s?

What about them makes for a good trade-off when considering the many other vendors?

We're not a K8s vendor. We're a lower-level platform than that. If all you care about is K8s, and no part of the rest of our platform is interesting to you --- the global distribution and Anycast, the fly-proxy features, the Machines API --- we're not a natural fit for what you're doing.

We were surprised at how FKS turned out, which is part of why we decided to launch it as a feature and all of why we wrote it up this way. That's all.

> We're not a K8s vendor

It might now be more accurate to say "You were not a k8s vendor", but now you are based on

> If K8s is important for your project, and that’s all that’s been holding you back from trying out Fly.io, we’ve spent the past several months building something for you.

If it's fundamentally different, maybe you shouldn't call it Kubernetes, perhaps a Kubernetes API compatible alternative?

fwiw/context, I use GKE and also many of the low-level services on GCP

Is Fly supposed to be simpler for the average developer?

Are any of the cloud provided managed K8s offerings just K8s under the covers? I’ve always assumed all of them shim the K8s api onto other more bespoke orchestration systems.

Most are pretty much actually k8s, though they tend to have different ways of handling the masters, my understanding is that it's the same k8s binaries and code

What I've seen is the k8s APIs working their way into other systems like cloud functions and servers, so you can use the same Yaml across vendors and products. This is further solidifying k8s APIs as an industry standard

Searching "managed kubernetes providers" is a good starting point to learn about the various offerings

I’ve looked extensively at the documentation for gke autopilot (a system I use extensively) and haven’t found any documentation on how they orchestrate those clusters under the covers.

I’ve always assumed it was a plugin to borg, not a different fleet orchestrator. Not to be overtly needy, but do you have a link to docs that contradicts that?

It is definitely not a plugin for borg. Borg is totally different api (source - i was borg sre). Afaik it’s actually vanilla k8s apiserver with some shimmed bespoke storage but it’s not really documented anywhere. You can test that fact using kubectl proxy though

- https://cloud.google.com/kubernetes-engine/docs/concepts/clu...

- https://cloud.google.com/kubernetes-engine/docs/concepts/kub...

Each of these has more peer pages that are pretty good too

You’ll note those documents are extremely careful in describing the control plane architecture to not promise you are running a stock k8s install. Which is why I’ve always assumed otherwise.

But I’ll trust the sibling comment which suggests the only bespoke component in gke is storage well enough to leave it alone.

Maybe you are reading it differently than me, but when they refer to the same binaries, as if I managed it myself, as being the pieces they use, it definitely seems like the open source project being used.

> The control plane is the unified endpoint for your cluster. You interact with the control plane through Kubernetes API calls. The control plane runs the Kubernetes API server process (kube-apiserver) to handle API requests.

> A node runs the services necessary to support the containers that make up your cluster's workloads. These include the runtime and the Kubernetes node agent (kubelet)

We are well into the reeds of what doesn’t matter, only in that fly has given us a very under the covers look at their implementation that is hard to find with other alternatives. But as someone who has run K8s in other contexts I find the following to be pretty circumspect (not in a way that causes me concern, I’m a happy gke user)

> GKE Autopilot manages the entire underlying infrastructure of clusters, including the control plane, nodes, and all system components. If you use GKE Standard mode, GKE manages the control plane and system components, and you manage the nodes.

There is a mile of implementation detail in that. Which I’m happy for them to keep on their side of the street.

> I’m a happy gke user && Which I’m happy for them to keep on their side of the street.

100% agreement

iirc, there was a time where I thought they were doing some consolidation things with how they run the control plane, then at some point my cluster updates had a warning related to control plane unavailability during an update, this was on a single node cluster

I get what you are saying though, there's probably some magic going on somewhere, but after many years on GKE, I don't really think about it.

I'm excited about this as a way to configure my Fly.io apps in a more declarative way. One of my biggest gripes about Fly.io is that there's a lightly documented bespoke config format to learn (fly.toml), and at the same time there's a ton of stuff you can't even do with that config file.

I love Kubernetes because the .yaml gives you have the entire story, but I'd _really_ love to get that experience w/o having to run Kubernetes. (Even in most managed k8s setups, I've found the need to run lots of non-managed things inside the cluster to make it user-friendly.)

Probably good for people already used to fly or interested in fly for other reasons, that could also use k8s ?

Sometimes you just want to run k8s without thinking too much about it, without having all the requirements that gcp have answers to.

If their reluctance were based on valid reasons that they handled in a unique way - might be good. In theory.

k8s has become a standard api and platform for running apps, having a * on it makes the implementation an outlier from the standard, not normally considered a good thing because you have to be aware of the nuanced differences.

Maybe a got fit for someone who is reluctant to use Kubernetes but has to for whatever reason.

If someone isn't a cloud provider they should be reluctant to use Kubernetes.

Aren't most k8s users not cloud providers?

It's more about good abstractions and APIs for running applications in the cloud. Cloud providers are the one's offering the APIs and abstractions we use, and increasingly putting k8s abst/apis at the forefront, because that is where industry has moved to

> Aren't most k8s users not cloud providers?

Yes. The point still stands.

People and companies use it because it makes this easier in several areas, it's industry standard at this point.

Can you explain why we should be reluctant to use k8s?

> People and companies use it because it makes this easier in several areas

Compute on demand is significantly more complex using k8s than what most companies already pay for from their own cloud provider, AWS is the industry standard for this purpose, followed by Azure, then TF, then Pulumi maybe.

k8s is a meme for resume-driven development.

Ok, you're obviously just a hater and aren't worth taking seriously

Plenty of companies/teams qualify as "cloud providers" even though all the usage is internal.

Reply on news.ycombinator.com