I used to maintain a couple of systems that between them were around 4 million lines of C++. Initially, in 2004, it would take about 16 hours to do a clean build. By the time I left that job, due to better hardware, precompiled headers and reduced dependencies, and smaller code for same functionality, clean build was down to 45 minutes.

The biggest winner, in my experience, was reducing dependencies in headers. Use forword declarations, etc. Precompiled headers helped a lot on Windows, but less so on Linux for the same code (but it still helped).

Incremental builds were fairly snappy, a minute or two (pretty good for a large C++ project).

I take it you haven't tried Swift ;)