I think this is what Rust does. No header files; #[cfg] syntax for conditional compilation for either source files or sections of code. But it doesn't seem to fundamentally help build times. xD (But maybe the bottlenecks in Rust are elsewhere.)
The problem with header files for me is the increased fragility because a change in a -D option to the compiler can potentially invalidate all existing targets but it might not and you cannot easily predict if it will. There is also potential for changes in the filesystem between builds to cause different headers to be found. So e.g. if I download a build done by you to help me not have to build a whole OS from scratch, it might easily end up totally rebuilding when it doesn't need to or not rebuilding when it should.
So I think probably as you said - Rust is winning in one area and losing badly in another. It might, however, be more consistent than C++ and less fragile and that would be an interesting characteristic.
FWIW on Symbian and Android I never got linear scaling - beyond 16 cores the benefits of more cores tailed off rapidly. I tried a lot of things and never worked out truly why. We were scheduling lots of processes and keeping the cores busy but tasks just took longer. I wonder about memory bandwidth but really don't know.