Worth mentioning other alternative small backends: http://c9x.me/compile/
If rust was a huge community okay, but face it, they are not. It is better therefore to focus their efforts where they can make a difference. A new x where the existing ones are just fine (this includes well maintained) is a waste of resources.
There are many possible good answers to the above question. However I'm not sure they apply, and worse I believe they will split resources that could be used to make something else better.
1) much easier for Rust community to contribute to the compiler from end-to-end.
2) lower coordination cost with LLVM giving complete, Rust-focussed control over code generation/optimisation. Think about e.g. fixing noalias.
3) lower maintenance cost for LLVM integration/fork.
It's also obvious that this needs to be weighed against the loss of LLVM accumulated technology and contributors. This is easy to underestimate (although I think 2)/3) are also easy to underestimate).
Sure it is harder to contribute to the backend, but does it matter? I've been doing c++ for years and never looked at the backend.
I'll grant lower coordination costs. However I believe they are not outweighed by the advantages of the other llvm contributions.
If they need to fork llvm that is a problem. Either merge it back in and be done (with some tests so whatever they need is not broke), or there is a compelling reason as llvm won't work with their changes.
You can file bug reports, but not every part of the project is going to receive the same level of attention or care from core developers, and not everyone has the same priority. For example the Glasgow Haskell Compiler had to post-process LLVM generated assembly for years because we lacked the ability to attach data directly next to functions in an object file (i.e. at an offset directly preceding the function). Not doing this resulted in serious, meaningful performance drops. That was only fixed because GHC developers, not any other LLVM users, fixed it after finding the situation untenable after so long. But it required feature design, coordination, and care like anything else and did not happen immediately. On the other hand the post-processing stuff was a huge hack and broke in somewhat strange ways. We had other priorities. In the end GHC, LLVM, and LLVM users benefitted, but it was not exactly ideal or easy, necessarily.
On the other hand, "normal" code generation bugs like register misallocation or whatever, caused by extreme cases, were occasionally fixed by upstream developers, or patches were merged quickly. But absolutely none of this was as simple as you think. LLVM is largely a toolchain designed for a C compiler, and things like this show. Rust has similarly stressed LLVM in interesting ways. Good luck if your language has interesting aliasing semantics! (I gave up on trying to integrate LLVM plugins into our build system so that the code generator could better understand e.g. stack and heap registers never aliased. That would have resulted in better code, but I gave up because it turns out writing and distributing plugins for random LLVM versions your users want to use isn't fun or easy, which is a direct result of LLVM's fast-moving release policy -- and it is objectively better to generate worse code if it's more reliable to do so, without question.)
Finally, LLVM's compilation time issues are very real. Almost every project that uses LLVM in my experience ends up having to either A) just accept the fact LLVM will probably eat up a non-negligible amount of the compilation time, or B) you have to spend a lot of time tuning the pass sets and finding the right set of passes that work based on your design and architecture (e.g. earlier passes outside of LLVM, in your own IR, might make later passes not very worth it). This isn't exactly LLVM's fault, basically, but it's worth keeping in mind. Even for GHC, a language with heavy "frontend complexity", you might suspect type checking or whatever would dwarf stuff -- but the LLVM backend measurably increased build times on large projects.
> Either merge it back in and be done
It's weird how you think coordination costs aren't a big deal and then immediately say afterwords "just merge it back in and be done". Yeah, that's how it works, definitely. You just email the patch and it gets accepted, every time. Just "merge it back in". Going to go out on a limb and say you've never actually done this kind of work before? For the record, Rust has maintained various levels of LLVM patches for years at this point. They may or may not maintain various ones now, but I wouldn't be surprised if still they did. Ebbs and flows.
I'm not saying LLVM isn't a good project, or that it is not worth using. It's a great project! If you're writing a compiler, you should think about it seriously. If I was writing a statically typed language it'd be my first choice unless my needs were extreme or exotic. But if you think the people working on this Rust backend are somehow unaware of what they're dealing with, or what problems they deal with, I'm going to go out on a limb and suggest that: they actually do understand the problem domain much, much better than you.
Based on my own experience, I strongly suspect this backend will not only be profitable in terms of compilation time, which is a serious and meaningful metric for users, but will also be more easily understood and grokked by the core developers. And Cranelift itself will benefit, which will extend into other Rust projects.
C++ isn't a great language, but learning C++ is the least difficult part of the problem to contributing to llvm.
So.. the claim that the Rust community is not big enough to achieve this is wrong, since they have already done it..
The reason they are doing it, is that LLVM is not fine: it is super _super_ slow. People want Rust to compile instantaneously, and are willing to pay people full time to work on that.
D, for example, compiles much faster than C and C++, and does this by having their own backend for unoptimized builds. I don't know how big the D community is, so I can't compare its size to the Rust community, but they did it, and it payed of for them big time, so I don't see why it wouldn't pay off for Rust as well.
Source? LLVM is fast for what it does.
What people usually complain about is rustc being slow overall, not the LLVM passes.
The LLVM phases are usually the dominating factor in Rust compile times (the other big single contender is the linking phase). However, when the Rust developers point this out, they are also careful to mention that this may be due to the rustc frontend generating suboptimal IR as input to LLVM; we can both acknowledge that LLVM is often the bottleneck for Rust compilation while also not framing it as a failure on LLVM's part (though at the same time it is uncontroversial to state that LLVM does err on the side of superior codegen versus minimal compilation time, hence the niche that alternative compilers like Cranelift seek to fill).
What I said was rust is better off focusing on problems that are not solved well by other people. A fast modern web browser (with whatever features is lacking) for example.
I think this works so well because language designers tend to understand compilers better than they understand other software.
JavaScriptCore does it differently: many of our benchmarks are either interpreters or compilers written in JavaScript.
One of those benchmarks, Air, is just the stack slot coloring phase of JSC’s FTL JIT (that JIT has >90 phases) rewritten in JavaScript instead of C++. It runs like 50x slower in JS than C++ even in JSC, which wins on that test. So, probably it won’t be possible to write a JS VM in JS anytime soon. I mean, surely it’ll be possible, but it’ll also be hilariously shitty.
It's extreme but it's a good idea because it treats compilation time like an actual budget, which it is. You can't just add things endlessly. But it's not easy to achieve in practice.
It would be nice if a gcc replacement compiler made speed of building code the goal. I'll even accept speed of building the compiler after it was compiled with gcc (clang, msvc...) as the benchmark if that is faster.