It's probably possible to produce a custom build for your machine that runs faster, whether by using custom instruction sets, or feeding different data to the optimizer when it's doing PGO. However, the risk you run there is that this could introduce bugs - Firefox and Chrome both regularly run into bugs caused by the optimizer or linker, and in some cases the fix is to stop the optimizer/linker from fiddling with that code.
No matter what tweaks I tried, the binary builds from Moz always ran better.
That being said, things like choosing the wrong preemption mode can introduce latency problems, etc. Most users experience higher throughput and higher latency when comparing Windows to Gentoo. I believe the higher user interface latency is the general case when going from Windows to Linux.