Why does compile times matter? 4 minutes? 4 seconds? Who cares?
Having to wait longer to see the results of one's work results in loss of focus and productivity.
I have worked on several projects where a full clean compile of the project takes 30+ minutes, and even incremental compiles and links to a large module will take 60-90s at absolute minimum because linker times are still shitty - and this is even with modern, powerful hardware, think 32 or 64 hyperthread Ryzen CPUs, not just your typical dev laptop.
It may not seem like much, but 60-90s is definitely enough to get you distracted and out of the zone.