Hacker News Reader

Comment by nickjj | Hacker News Reader

> As a platform for running production code it might be dead or dying, but as an ecosystem and a development tool it will continue to live and probably thrive.

Not everyone needs an over the top Kubernetes cluster in production. I'm plenty happy using Docker Compose in production and foresee myself continuing to use it as long as Docker maintains it. There's even a WIP issue on their roadmap[0] to rewrite Docker Compose with Go to make it more consistent with their main CLI. Looks like full steam ahead to me rather than death.

It's really nice having the same Dockerfile + docker-compose.yml get used in dev + ci and prod. There's no surprises and no massive amount of complexity.

[0]: https://github.com/docker/roadmap/issues/15

View on HN

I second this, we've been running Docker Compose in production for years and love it. It's massively simplified our production environment, and for a company that only serves 100,000-1,000,000 monthly hits it's the perfect middle ground. We've tried k8s several times and have always ended up going back because we didn't need the complexity.

We even run compose v2.4 to retain the ability to set container memory/cpu limits and define startup order with healthchecks because we don't need the complexity of swarm and v2.4 provides everything we need. Sometimes simple is best.

I personally use compose to manage services on my own laptop and homelab as well, it just has the massive advantage of being the greatest common denominator that everyone can pick up in a few days if they're familiar with basic container concepts.

Couldn't agree more. compose is a wonderful tool that lets you avoid the k8s complexity if you don't need it.

Exactly this here as well. I have helped setup a few successful SMEs with Docker, Compose, and some CI/CD best practices. They are running 7-figure businesses on the backend.

>>> We even run compose v2.4 to retain the ability to set container memory/cpu limits and define startup order with healthchecks...

Does compose finally support starting containers in order with healthchecks? Last time I checked it didn't and Docker Inc was expressively refusing to support that, in spite of being a very basic use case.

See various workarounds over the years: https://stackoverflow.com/questions/31746182/docker-compose-...

Compose v2.4 was released long ago and abandoned by docker. Some features/workarounds that worked in that version were killed in the next versions. It seems to me that you're stuck on that old version unable to upgrade because of that?

IMO this is a perfect illustration of how Docker sucked in practice (both the product and the company), the industry naturally converged into trying to get rid of it because there was no other choice.

You just define a depends_on: condition: service_healthy, works great in v2.4. v3 is really a different beast, it's not an upgrade in the traditional sense, more of a lateral move with the introduction of swarm. I've even seen Docker employees recommend in forums using v2 independently of v3 if you need certain features (I can try and dig up a link from my archive if you're interested).

Compose file version 2 is not deprecated, unlike version 1. It remains stable and supported (though no new features are added), it's very well documented and still widely used because of the additional features it supports over v3. https://docs.docker.com/compose/compose-file/compose-version... (compare with the v1 deprecation notice above it)

I don't know about "sucked in practice", we use it because we love it. They released a new version to push users toward swarm, which we don't need. Such is life sometimes. It's had some bugs and problems in the past and if something better comes along we'll use it, but for the time being Docker compose satisfies all our needs and I think it's a beautifully designed UX and reliable piece of software in its current form.

Problem being, "depends_on" was removed in version 3, with no replacement in sight.

I honestly don't understand why they'd remove that? why make a new configuration format just to remove critical options? what is the developer supposed to do instead? when will compose stop accepting the v2 format so we're definitely screwed for good? (yes the writing is on the wall).

P.S. For readers who don't have context. Make a basic compose file with a webserver and a database. "docker-compose up" is failing half the time out-of-the-box because the web server starts before the database.

> "docker-compose up" is failing half the time out-of-the-box because the web server starts before the database.

Most web frameworks will retry until it can connect or give up after a specified timeout time. Basically handle it at the app layer where you have the most control.

But you still run into situations where you need to wait for docker-compose up -d to be "really" up before doing certain things (such as exec'ing into your main container to run tests or migrations).

For that I wrote a simple Bash script at https://github.com/nickjj/wait-until. IMO a problem like this can't really be solved at the Docker Compose level. But without such a script you can easily end up with failing CI tests because there's a multi-second but varied amount of delay on upping a database for the first time. That script has become a part of all of my CI / deployment pipelines.

Wait for a TCP port to come up or wait for a HTTP healthcheck to succeed. I think it's exactly the sort of things that should be supported by the tool out of the box. It's a very common need.

The job of the tool is to manage a bunch of containers that depend on one another and restart them on failure. The tool has to know about dependencies and liveness anyway.

Docker compose development was compromised by the agenda of pushing people to use swarm.

This is why v2 features like resource limits were moved under the deploy: key and you have to use the special "--compatibility" flag to get them to work locally.

This is no longer the case. Compose is an open spec and the v2 and v3 formats are being merged into a unified format (where you don't need version numbers). There are now multiple implementations (eg for ECS/Fargate and ACI), and implementations are being encouraged to support all the options.

This is an application-level design flaw, not a flaw inherent to docker-compose. Regardless of whether you use docker or even if you use containers at all, your webapp should be able to restart if the connection to the database fails or is interrupted for any reason.

You would have the same problem if you used supervisord for example. The general solution is to auto-retry the connection until it comes back up, or hard fail and auto-restart the entire app in the event of a hard failure (e.g. with restart: on-failure or autorestart=true).

Hm, yes, but to play devil's advocate - if you want compose for orchestration (I don't) then you want that dependence not so much for initial start up, but for triggering restarts on failure.

As it stands, AIUI, all it does is mean that dependency A comes up too (in some order) when you `up B`.

You can define any order or restart behavior you need with v2. As long any container with a service dependency fail hards if that dependency is missing/down, it will be restarted and the whole app will eventually reach an up state.

    restart: on-failure
    depends_on:
      other_service:
        condition: service_healthy

https://docs.docker.com/compose/compose-file/compose-file-v2...

From GP:

> Problem being, "depends_on" was removed in version 3, with no replacement in sight.

How is v2 relevant?

v2 and v3 specs are merged as of September apparently, so either set of options works in the current version. (See cousin comments above)

As of the September release of Compose version 1.27.0 https://github.com/docker/compose/releases/tag/1.27.0 the config format versions 2 and 3 are merged and "depends_on" – and "extends"! – are back.

This is amazing news, I was not aware of this. Thank you so much for linking! Do you know if it also applies to the cpu/memory limits and health checks? Is it a true full merge of the two specs? (I will investigate myself and post back too)

I use Docker Swarm (basically multi-node Compose) in production, and am also really happy with it. It's an order of magnitude simpler than k8s, and perfect for small-medium scale deployments.

In terms of configuring your stack, you use regular Compose files, with a few enhancements available (such as better support for secrets). I love it!

I also want use docker swarm in production, but I keep hearing people say it has network bugs like after some time services cannot talk to each other. Have you experienced any such issue?

I haven't experienced any networking issues in production at all.

I have experienced 1 networking issue, but only specifically when keepalives are disabled on Windows (where I use Docker for dev/test only)[0]. When using an overlay network, network connections to dockerised Postgres go "stale" after 15 minutes. I workaround it by publish the Postgres port in "host" mode instead of the default of "ingress" mode using `endpoint_mode: dnsrr`.

[0] https://success.mirantis.com/article/ipvs-connection-timeout...

What's your solution to the problem of shared disk between hosts ?

Nfs volume mounts. Works quite well. Make sure the hosts can access the nfs endpoints (i run nfs on separate lan)

I've never actually had to use shared disk, but I'd use NFS if I did (as another commenter suggested) - I've tried it out before, just when playing, and it worked fine.

I recommend taking a look at Hashicorp's Nomad. I was in a similar situation where I wanted container orchestration but our use case wasn't yet at the point where the complexity of kubernetes was justified. Nomad strikes a great balance. Its single binary and makes it easy to get a cluster up and running. You can run tasks other than docker containers as well.

Compose plus VS Code’s remote container plugin gives a brilliant development env, and easily deployed to production. I agree with your assessment!

Reply on news.ycombinator.com