Hacker News Reader

Comment by kh_hk | Hacker News Reader

docker compose is unnecessarily nerfed for using in development. I swear every company I work with has this problem and I always end up carrying along my hybrid bash-to-docker-compose cmd line tool that makes the dev experience under compose nice.

Did you know that compose accepts stdin for `-f` files? Anything that outputs yaml is a valid "compose provider". Self plug, more on that, https://eskerda.com/complex-docker-compose-templates-using-b...

View on HN

So whats the complaint? Compose seems pretty easy to work with to me.

For me the problem is that yaml (and the compose specificatio n) is not self sufficient to support complex scenarios. You will start by setting things and discovering that ENV vars * on the global scope of the compose specification are not supported so you end up having to template and output yaml.

Try having 40 microservices and quickly iterate between fixing bugs in them. Some you want mounted, others you want running unmounted. I understand the premise is already broken, you do not want 40 microservices, but that's not a precondition you can fix without a time machine.

[[* EDIT: by env vars on the global scope, I mean setting things on the global scope by using an env var, say, you want a certain volume named after an environment variable. You can only use ENVs as values.

    # this works
    foo: "${SOME_ENV}"
    # this does not
    ${SOME_ENV}: "foobar"

Your problem is that you can't set ENV variables at the global level? Have you looked at YAML anchors? Makes things more DRY for longer/complex compose files.

That's just the tip of the iceberg. My problem is that YAML is not a turing complete language (and dev environments are usually messy by their nature).

Given that YAML it's a configuration language I'd say that not being turing complete it's a feature, not a bug nor a limitation; I always want my language files to be declarative to not suffer the perils of logic.

Edit: Also, I don't see the need for a turing complete language for something like docker compose, if you need something really complex you can always script a docker-compose.yml generator with all the logic and complexity you need.

You might find Cog (https://nedbatchelder.com/code/cog) helpful for generating YAML on the fly.

When I need a Turing-complete Compose file, I use Pulumi to spin up the Docker containers and wire things together: https://www.pulumi.com/blog/pulumi-and-docker-development-to...

This is literally ‘why kubernetes?’

Docker-compose just works and my team gets shit done. Does your team get shit done? If they do then what are you complaining about? If it hits production then it’s all good!

All I am saying is:

* docker compose could be better

* docker compose does not improve that much

* by mixing docker compose with templates you can get complex stuff going, but is not a much advertised feature.

Most of the "cool dev tool env" that appear are based on kubernetes, or something that is _not_ docker compose, which makes them difficult to adopt. I would like docker compose to improve on a direction that makes development easier, that's all.

You can accomplish the same thing using environment variables. You can pass them in at build time or deploy time. `ENVIRONMENT=dev docker-compose up` or `docker-compose build --build-arg ENVIRONMENT=dev` and in your case maybe you write some function `find_answer() { something-async ... }; ENVIRONMENT=$(find_answer) docker-compose ...` You can also read in .env files... Which in my opinion is cleaner since you can check the configurations into source control for YAML or .env files which is easier to read than hundreds of lines of bash and it's esoteric syntax. I love bash but I'm just sayin.

It's a simple example, maybe too simple to get the point. What I want is to conditionally set things on a yaml file. If you have ever used a templating language, you know this problem

    services:
        redis:
            ...
    
        {{ if some_condition }}
            volumes:
                bla bla
        {{ end if }}
    
        {{ if some_other_condition }}
        another_service:
            ...
        {{ end if }}
    
    volumes:
        {{ for volume in volumes }}
            - ...
        {{ end for }}

Podman allows for Kubernetes templates, so theoretically you could use Helm for this potentially.. otherwise, Minikube on Podman. I wish they had Go templating in the compose files though, that would be rad.

no haha I get the point but it's just another example I see someone over engineering a problem where it should be declarative. In what use case would you dynamically provision some arbitrary n number of volumes without knowing the path names to them? for volumes why wouldn't you just use "volume_name:/some/path" ... "another_volume:/another/path" and let docker manage the volume for you? you're going to end up writing in a different file "/some/path,/another/path" so why not just put it in the compose file?Also why wouldn't you just mount the root path and put n paths inside of them instead of making them all volumes? I'm struggling to see that you're use case actually needs such a feature and you're solving a problem for something that's actually broken somewhere else. you're adding some virtual layer that's making things harder to manage or understand because you don't understand all the features of docker compose technology instead of fixing the root of the issue. I'm not convinced that you're "feature" is necessary.

Everything is simple until someone asks you to try and diagnose (and hopefully fix) a bug on an upgrade between two different versions of a piece of software that can be deployed using 5 different strategies or that has 20 moving services...

your volume example is an anti-pattern in regard to something like the 12 factor app. you're conditionally changing and setting the volumes based on an environment your in? your code and service is gonna be busted. how would you even know if anything worked in prod if it's not reproducible in dev since # if dev then volume xyz else volume abc #

It's an anti-pattern just because you say so. This is a simplified example of a complex problem that needs arbitrary volume mounts. Using volume mounts to run software inside containers is a common dev pattern on scenarios where rebuilding takes too much time to quickly iterate. I am sorry, but unfortunately I cannot expand the issue further here to help you understand it, and I am happy too if you think I am over-engineering compose.

I honestly don't care if it's you misunderstanding or your teammates are building a bad framework/foundation to iterate upon. Doesn't matter to me either way bud. But you're going against the premise of docker and docker-compose which is write once and run anywhere because the services will be different every time you deploy since they'll have have variable path and file locations etc. - that's why it's an anti-pattern not because "i say so"

the point of docker-compose is partly so that your stack is ephemeral? you stand things up and shut it down with one simple switch and it can move between different environments and use the same declaration as the last environment. so now I gotta drag your bash script around and embed my yaml file in it? lol

The point of docker compose is to orchestrate containers. It's great for you to have lines that you always color inside, but other people choose different lines.

you're coloring outside the lines and making a mess you should grab a different coloring book this ones not for you.

Why do you want to set things conditionally like that?

This feels like it's a very incorrect solution to a problem that you're making far more complicated than it needs to be.

I apply this solution exclusively on the development environment. I have found that having the power to bail out to a shell whenever docker compose does not support something helps me iterate faster on my local dev (without affecting others).

The following is just a contrived example. Why would I do that? Because I can

    # Bring up services without a command
    #
    # usage:
    # nocmd.sh up -d service_name
    
    function _compose {
      docker-compose -f docker-compose.yml $@
    }
    
    function nocmd {
    cat << EOF
      services:
        $1:
          command: tail -f /dev/null
    EOF
    }
    
    function nocmd_compose {
      local service=${@: -1}
      _compose -f <(nocmd "$service") $@
    }
    
    nocmd_compose $@

I can punch myself in the dick but I don't

The compose file is declarative for clarity, so instead of fighting it, do the things you want to do at other levels. Variables? Pass in some env vars. Conditionals? You can specify multiple configuration files so that whatever calls `docker compose up` can deal with the conditionals of deciding which config files to include.

Reply on news.ycombinator.com