1 package per layer can actually be quite nice, since it means that any package updates will only affect that layer, meaning that downloading container updates will use much less network bandwidth. This is nice for things like bootc [0] that are deployed on the "edge", but less useful for things deployed in a well-connected server farm.
It's called a layer because each layer on top depends on the layers below.
If you change the package defined in the bottom most layer, all 49 above it are invalid and need re-pulled or re-built.
Docker exploits this to figure out when it can cache a layer, but building a container is different than running one because changing the underlying file system can change what a command outputs. If you're running a container, changing one deeply buried layer doesn't change the layers above it because they're already saved.
I also initially thought that that was the case, but some tools are able to work around that [0] [1] [2]. I have no idea how it works, but it works pretty well in my experience.
[0]: https://github.com/hhd-dev/rechunk/
[1]: https://coreos.github.io/rpm-ostree/container/#creating-chun...