Hacker News Reader

Comment by cbushko | Hacker News Reader

There are several reasons:

  - Google has a lot of experience running containerized services because of Borg. Google Research says that they have been running these workloads since 2005.
  - The above gives them insight into how to do this well. They would have the internal infrastructure, logging and monitoring already setup.
  - They are the creator and still a major contributor to Kubernetes itself which means they can insert their influence on it.

To be fair, my only experience running Kubernetes is on GCP so maybe they are all this easy to use and run. The internet tells me that other offerings are not as good. Compared to running workloads on EC2 instances and ECS, using GKE is amazing and pain free.

Note: I am responsible for 5 clusters, 300+ nodes total and several thousand running services. Not huge by any means.

View on HN

The Borg thing is pure marketing.

Amazon also has been running internally on containers for years before ECS. But workloads for massive FAANG companies designed by FAANG engineers turn out to be quite different than most AWS/GCP customers.

Just like EC2 wasn't Amazon selling its "spare capacity during off-peak" but always a purpose built service with completely isolated data centers and network fabric from day 1, the connection between Borg and Kubernetes is tenuous at best.

Honestly, I would encourage you to evaluate ECS and Fargate. Forget Kubernetes and direct instance management. Try a construct like https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws... and see how much simpler it is than all the K8S overhead.

Source: I'm at Amazon but I work on none of these services and opinions are my own. I played with K8S and am eternally confused how it's as popular as it is. I guess it gives you the illusion of not having lock in?

1. Is instance management on ECS really annoying enough to justify the higher fargate price?

2. Does ECS have something similar to pods, i.e. colocated containers which can communicate over localhost, share filesystems, etc? A quick search didn't turn anything up.

I only have one thing to point out about Borg and that is large numbers of Borg developers started working on Kubernetes. They got rid of Borg mistakes and replaced them with all new mistakes. :)

I don't see K8S being about lock-in at all. Using a cloud provider gives you some sort of lock-in and you should be utilizing that providers strengths.

Using Kubernetes is all about the patterns that it provides and how it removes a certain class of problems for you. Deployments, scaling, logging, etc are some of the patterns it provides and the consistency matters. How many of us have worked at companies where deploying two services have been completely different? One team runs the jenkins pipeline while another ftps the files over. Now multiply that by several services and several tasks (logging, scaling, etc).

The benefit is in the patterns.

I was hoping for something more concrete, like limitations or incidents and not just "Google has more Kubernetes experience".

What were your pain points with ECS? Did you use ECS with or without fargate? What about EKS, since that sounds like a closer equivalent to GKE.

That's fair. As I pointed out in another reply, most of my experience is with GKE so I may be spoiled in how painless it is. All I can say is that GKE is treating me well. Node upgrades are seemless, defaults are decent, it was very easy to create clusters through the console or terraform. I only run 5 clusters and that is not that huge compared to others.

I've used ECS more than fargate. My biggest problem comes from reliability and having to manage things. We still run things on ECS and we get about 30x as many maintenance things we need to do, like retiring instances or instances just dying on us and we have to move things off of it manually.

I have a little bit of experience in Fargate from a previous job but not enough to make a real conclusion about it.

I've never used EKS but I do have friends that would rather roll their own version of Kubernetes on EC2 instance instead of using EKS.

Reply on news.ycombinator.com