From my observations multi-threading the linker would be more useful, since for small changes in bigger projects the linking step usually takes a lot longer than compiling a couple units.
On the topic of linking-- can you sub in the gold linker for faster linking in your projects?
Linker slowness ~= GC pause. The cost is approximately proportional to the size of the traced graph.