Configuration changes. With immutable image every time any parameter changes you have to rebuild and re-deploy image which looks like a waste of resources - to change even 1 byte you need to re-build and re-push an image (which could be a multi Gb one). Then restart a whole cluster which depending on the application can be either costly or disruptive (in case of long lived network connection, on in case if you have a database). And sometimes you need to change configuration parameters many times per day.

What developers tend to do when forced to use immutable infra is to move configuration from on-disk files to RAM and query it using network API from a central system. The problem is that it makes systems less reliable. If a VM/server restarted it can practically always read a config form disk but if you service relies on an external system to get runtime configuration it would not work if this external system is down, overloaded, misconfigured, returns wrong config because of a bug e. t. c. And it does happen in practice even when system designers tell they configuration API is very reliable (in theory). After seeing such systems fail I like simple on-disk configs more and more.

Sure. And some of us may want or need to do live changes at runtime across the fleet without needing a full rebuild and redeploy. It can certainly be more green. It's fine if you find it useful to be dogmatic and consistent but don't expect that to be the right approach everywhere or for everyone.

CM is useful here. And usually in the bootstrapping of such architectures.

Er, and how to you make the images? Or a family of tweaked images for different use cases. The make sure security standards, access to centralized services, etc is working?

Doesn't that just move where you need CM a bit?

Yeah, I have used that approach since around 2015. In order to create these prebaked images, you will usually want to use a CM tool anyway, though.