Hacker News Reader

Comment by marcus_holmes | Hacker News Reader

I still don't get the benefit of running Go binaries in containers. Totally get it for Rails, Python, etc, where the program needs a consistent set of supporting libraries. But Go doesn't need that. Especially now we can embed whole file systems into the actual binary.

I've been looking at going in the other direction and using a Go binary with a unikernel; the machine just runs the binary and nothing else. I haven't got this working to my satisfaction yet - it works but there's still too much infrastructure, and deployment is still "fun". But I think this is way more interesting for Go deployments than using containers.

View on HN

As the author of Caddy, I have often wondered why people run it in containers. The feedback I keep hearing is basically workflow/ecosystem lock-in. Everything else uses containers, so the stuff that doesn't need containers needs them now, too.

Love your work :) Good to know I'm not totally out there on this :)

For my personal site I run Caddy in a Docker container along with other containers using a compose file. By doing it this way, getting things up when moving to a new instance is as simple as running `docker compose up`. Also making changes to the config or upgrading Caddy version on deployments is the same as a other services since they're all containers. So it's easy to add CI/CD and have it re-deploy Caddy whenever the config changes and there's no need for extra GitHub Actions .yaml's. Setup as code like this also documents all the dependencies and I think it might be helpful in the future.

Having said that, for serious business, this setup doesn't make sense. It possibly takes more work to operate as container when the gateway runs on a dedicated instance.

I find putting http proxies in containers to be a very effective method of building interesting dynamic L7 dataplanes on orchestrators like k8s. Packaging applications (particularly modern static SPAs), with a webserver embedded, is also a very intuitive way of plugging them into the leaves of this topology while abstracting away a lot of the connectivity policy from the app.

Of course there's also the well-known correlation between the quality of your k8s deployment and the number of proxies it hosts. /s

We just gave this a go (pun intended) on Unikraft / kraft.cloud , Caddy runs fine on it!

Containers can give you better isolation between the application and the host, as well as making horizontal scaling easier. It's also a must if you are using a system like Kubernetes. If you are running multiple applications in the same host, you also have control over resource limits.

Kubernetes requires a platform runtime that answers its requests, but there's no law enforcement agency that will prevent you from using a custom runtime that ignores the very existence of Linux control groups.

Yes, that is true. Though if you are using Google's or Amazon's managed Kubernetes services, I think you need to use Docker.

Certainly EKS is less managed than it appears, I’m quite confident that a node running something implementing the kubelet API convincingly would work. They changed recently (1.23 maybe) from Docker to containerd.

Go programs still make use of, e.g., /etc/resolv.conf and /etc/ssl/certs/ca-certificates.crt. These aren't strictly necessary, but using them makes it easier--more consistent, more transparent--to configure these resource across different languages and frameworks.

I use the semi-official Acme package [0], which handles all that really well. I haven't touched SSL or TLS config for years. I mean, I might be an outlier, but this seems pretty standard for Go deployments these days.

[0] https://pkg.go.dev/golang.org/x/crypto/acme

Hi, one of the maintainers of the Unikraft LF OSS project here (www.unikraf.org), clearly in agreement :) . We regularly run Go workloads, and even use Dockerfiles to build Go and other projects that we then turn into minimal unikernels. We also have a closed beta/free (unikernel) cloud platform if people want to try, sign up is at kraft.cloud .

A lot of companies are Kubernetes based now so containers are the default delivery mechanism for all software.

Here's what a container gives you:

  - Isolation of networking
  - Isolation of process namespace
  - Isolation of filesystem namespace
  - CPU and Memory limit enforcement
  - A near-universal format for packaging, distributing, and running applications, with metadata, and support for multiple architectures
  - A simple method for declaring a multi-step build process
  - Other stuff I don't remember

You're certainly welcome to go without them, but over time, everybody ends up needing at least one of the features containers bring. I get the aversion to adding more complexity, but complexity has this annoying habit of being useful.

A VM provides the first 4 anyway - if you're deploying to a cloud instance then having these in the container is redundant. If you're deploying to bare metal then it's possibly useful, but only if you're deploying multiple containers to the same machine.

Go doesn't need a format for packaging - it's one file. It's becoming common practice to embed everything else into the binary. (side note: I haven't done this with env files yet, and tend to deploy them separately, but I don't see any reason why we don't do this and produce binaries targetted at the specific deployment environments. I might give it a go).

I kinda prefer makefiles for the build stuff, or even just a script. The whole process of creating a Docker instance, pushing source files to it, trigging go build and then pulling back the binary seems redundant; there's no advantage of doing this in a container over doing it on the local machine. And it's a lot faster on the local machine.

Talking to people, it appears to be as mholt said: everyone just does everything in containers so apparently we do this too.

Are you suggesting to setup a separate VM for each process that may only require like 0.25 cpu? Another thing you can’t do with VMs is oversubscribe (at least not with cloud ones)

You can't have both oversubscription and isolation, almost by definition. If you want isolation, VMs are great. If you want oversubscription, OSes are still better than container runtimes at managing competing processes on the same host.

OK but I can tho - oversub cpu, isolate memory, systems resources (like ports) etc.

> OSes are still better than container runtimes at managing competing processes on the same host.

OSes and container runtimes are the same thing

> OSes and container runtimes are the same thing

For a subset of OSes

Go binaries tend not to be that lightweight, because we have goroutines for that.

And yes, setting up a separate VM for each instance of a process is perfectly feasible. That's what all this cloud business was about in the first place.

This goes against everything we've learned about effectively deploying and managing software at runtime. Using the golang binary as a packaging format for your app has the same energy as crafting it exclusively from impenetrable one-liners.

Sorry, I don't understand this. What's the difference between a Docker image made from a script and a Go binary made from a script?

The Docker image is more useful

I think we're back to square one here. Yes for other languages the Docker image is more useful, but for Go the image just contains a single binary file. It's just more bloat rather than more utility.

At Unikraft (OSS unikernel project) we do a bit of both: if wanted, people can specify via a Dockerfile what they want/need in their filesystem, and then we have a tool that compiles (if needed) and packs the files into a (OCI formatted) unikernel (which we then deploy via kraft.cloud ).

Reply on news.ycombinator.com