Would multi-stage Docker builds not help here? Composer executes in one step and the result artefacts are copied into a "clean" PHP image without Composer installed.
But in development it's much easier to have it in the image. Additionally we do not bundle the code in the dev image but bind mount it in Docker Compose, which is much faster than rebuilding the image to test changes in development; PHP not being compiled allows us to do that to reduce the feedback loop duration.