However, building the library set and examples and tests is a one-time cost for the non-header-only libraries. And you don't need to compile the examples or the tests. Or the various debug-release combinations.
Plus, why wouldn't you cross-compile, anyway? I do lots of embedded sensor work and I wouldn't think of compiling anything directly on the hardware itself.
What it amounted to was debugging horrible compiler crashes, broken library behaviour and spending a long time deciphering huge template names. This caused me to hate C++ more than you can ever imagine. I swear whoever decided to stitch templates together in the way Boost does has never had to debug a development compiler, this broke almost every part of the toolchain.
Ugh.