I think starting with LLVM was the right decision (and one that was I was primarily responsible for). Rust would lose most of its benefits if it didn't produce code with performance on par with C++. LLVM is not the fastest compiler in the world (though it's not like it's horribly slow either), but its optimization pipeline is unmatched. I don't see replicating LLVM's code quality as feasible without a large team and a decade of work. Middling code gen performance is an acceptable price to pay until we get Cranelift; the alternative, developing our own backend, would mean not being able to deploy Rust code at all in many scenarios.
https://www.phoronix.com/scan.php?page=article&item=gcc-clan...
Of course, benchmarks, yada yada, but at least is some sort of comparison axis where the improvement over the years is clear.
Typically, developers can afford to throw more cores and more ram at their build machines.