Fixed your link to the full article:
https://lwn.net/Articles/821367/OSTree is still working very well for us. At the time I wrote the article we had been using OSTree (and the build system I described in the article) for 4 years; 3 years later not much has changed. In those 3 years we have started building images for different architectures and our build system / OSTree handled it just fine, as you'd expect (it already handled cross-compilation, but for a single architecture).
Our build system also builds Docker containers (for our cloud services) but instead of a Dockerfile we use a custom yaml format that lists the base image (like "FROM" in a Dockerfile), the apt packages to install, and the commands to run (like "RUN"). Then we create a lockfile of all the apt packages, download them (into a local OSTree repository, but that's an implementation detail), and install them with a custom "docker run" command + "docker commit". We end up with the base layer + the apt layer + a single layer produced by concatenating all the RUN commands + a layer with our compiled Rust binaries and Python files. We use apt2ostree to generate the lockfile (really it's our patched version of aptly doing the work) but we use docker (not OSTree) to build the Docker layers. We use Docker's standard push/pull mechanisms to deploy these containers.
To hook this up to Ninja we use "marker files" (a file in the build directory) to track whether this work has been done (e.g. you need to regenerate the apt layer if that layer's "marker file" is older than the lockfile).
Our Python "configure" script (i.e. our build system) looks something like this:
def build_docker_container(name):
yamlfile = f"{name}/docker-base-image.yaml"
with copen(yamlfile) as f:
data = yaml.safe_load(f)
ubuntu_version = data["ubuntu_version"]
image = docker_pull("ubuntu:{ubuntu_version}")
image = docker_apt_install(image, data.get("apt_dependencies"),
lockfile="%s/Packages-%s-amd64.lock" % (
name, ubuntu_version),
ubuntu_version=ubuntu_version)
cmd = "..." # RUN commands from yaml
deps = [...] # dependencies from files listed in yaml
return docker_mod(image, cmd, deps)
(Where "copen" is like "open" but it tracks which files have been read by "configure" itself, to detect if we need to re-run "configure").
What I really like about the Python+Ninja combo is that you can pass these targets around as Python variables — you don't have to come up with an explicit filename for each one (the target name is generated by each helper function, e.g. `docker_pull` returns "_build/docker/${name}"). This makes it so convenient to compose these build rules. And if you ever need to debug your build system, everything is very explicit in the generated ninja file.
We don't have a ton of these containers so it has worked well enough for us because the apt layer changes rarely, and the layer above it is small/fast. The main thing we get (that you don't get from vanilla docker/apt) is lockfiles and reproducibility.