It's unfortunate that Boost is an all-or-nothing set of libraries, and the meta-programming in there can be brutal to compile, so for that kind of platform you'd really need to weight the pros and cons of what you're using out of Boost vs. compile times.

However, building the library set and examples and tests is a one-time cost for the non-header-only libraries. And you don't need to compile the examples or the tests. Or the various debug-release combinations.

Plus, why wouldn't you cross-compile, anyway? I do lots of embedded sensor work and I wouldn't think of compiling anything directly on the hardware itself.