Hacker News Reader

Comment by mysterydip | Hacker News Reader

Makes me wonder about a distributed compilation. If everyone had a monster workstation but not everyone compiles at the same time, a theoretical networked compiler (which may already exist as I haven't really checked) could spread the files out among available workstations and bring things back together near the end.

As the largest issue is the throwing away of duplicate work, I'd see it as a kind of reverse binary tree: machines working on files that depend on others talk together, then when finished send the condensed work up the chain (and signal their availability for the next workload chunk or phase) until everything collapses down back to the original machine.

View on HN

https://wiki.archlinux.org/index.php/Distcc

I used that 10+ years ago on Gentoo, and never saw anyone using it since. Don't know how often is used now days.

I used it ~3 years ago on Funtoo; there's definitely still people that use it!

distcc: https://en.wikipedia.org/wiki/Distcc

The best solution today for distributed build is: http://www.fastbuild.org/docs/home.html

It is faster than IncrediBuild, even faster than SN-DBS and has multi-platform support, the only problem is that it requires its own build script.

Ok, I see distcc helping people with computer labs and server farms, but I thought mysterydip wanted to help the Gentoo community and other such. I.e. peer-to-peer build sharing.

So what kind of cryptographic guarantees would you need for that? And if you can only verify the build results by trusting signatures from upon high, then what is the point? Perhaps those builds could be turned into work in a proof-of-work blockchain. Do compilers contain any hard-to-do, but easy to verify steps?

Whole shelves full of useless PhDs thesis are just waiting to be written on this topic.

Isn't this what IncrediBuild does on Windows?

It is. Although, as far as I'm informed (having last used it in 2013 or so), even with that approach, you'll likely have problems linking all the output in the end if your project is large enough.

Of course, thats another separate problem to begin with. I still remember dabbling with D and vibe.d and replacing the default GNU linker with ld.gold because over 90% of the build time was due to the linker...

Reply on news.ycombinator.com