Bigger ASM means you can fit less code in cache, which makes your code slower. If this happens in a hot loop, it may be the difference between "my code fits entirely in L1 cache" and "my code needs to be fetched from L2/L3".

This matters in high-performance scenarios. The fact that you can introduce a performance issue simply by creating a temporary is absolutely bonkers, and makes writing high-performance code more complicated than it needs to be (and god knows it's already very complicated).