What is the alternative that is better? The ability to have layers that build on top of each other and can be cached is a big feature... what alternatives provide that and are better?
Taken to the extreme, look at something like Nix (or conda, come to think of it). Why can’t I just have one copy of a package of a given version shared by all containers, if they all want that package? Unix file systems should be great at that kind of composibility; that’s the advantage of a unified tree instead of a tree-per-source. But in the docker model, you’re stuck with a stack.
My ideal image definition is a hybrid between docker’s immutable hash-addressed image layers and an fstab file to describe how and where to mount them all.
Unfortunately, those behaviours are mutually exclusive with transparent layering.
It's certainly possible to build a file-system whose behaviours are compatible with that kind of transparent layering - Plan9 was built on exactly that model, for example - but then it wouldn't be a POSIX-compliant filesystem anymore.
The promise of Docker was that you'd be able to deploy your existing applications in a more reliable, repeatable way, but that breaks down when you have to tinker with your application's file-handling code, or jump through extra hoops to flatten the layers of your container's filesystem image.
* The general idea of mixing together filesystems+folders to achieve re-use/sharing/caching.
* The "Dockerfile" approach to this - with its linear sequence of build-steps that map to a linear set of overlays (where each overlay depends on its predecessor).
The "Dockerfile" approach is pretty brilliant in a few ways. It's very learnable. You don't need to understand much in order to get some value. It's compatible many different distribution systems (apt-get, yum, npm, et al).
But although it's _compatible_ with many, I wouldn't say it's _particularly good_ for any one. Think of each distribution-system -- they all have a native cache mechanism and distribution infrastructure. For all of them, Dockerization makes the cache-efficacy worse. For decent caching, you have to apply some adhoc adaptations/compromises. (Your image-distribution infra also winds up as a duplicate of the underlying pkg-distribution infra.)
Here's an alternative that should do a better job of re-use/sharing/caching. It integrates the image-builder with the package-manager:
https://grahamc.com/blog/nix-and-layered-docker-images/
Of course, it trades-away the genericness of a "Dockerfile', and it no doubt required a lot of work to write. But if you compare it to the default behavior or to adhoc adaptations, this one should provide better cache-efficacy.
(All this is from POV of someone doing continuous-integration. If you're a downstream user who fetches 1-4 published image every year, then you're just downloading a big blob -- and the caching-layering stuff is kind of irrelevant.)