You also can often speed up I/O that way since you can saturate both your I/O and CPU bandwidth, rather than alternating between waiting on them.
I also take it that you're not compiling to individual object files since usually compilation since usual compile parallelization is based on farming out single object file compilations to multiple machines or processes.
I wrote some small portions of "icecream" which is used for compilation parallelization by the guys at SUSE and there it would actually transmit over the network a full chroot environment so that even different Linux installs working on compatible architectures (later cross-compiler support was added as well) could execute the compiler jobs of rather different OSes.
"Hmm... Love to get one of those new SSD drives. Just need to justify it somehow"