As a longtime follower of Rust development, I can say for certain that language-level features have never been rejected for being "too tough too implement". Quite the opposite, in fact: major compiler refactorings in order to improve the language happen regularly (e.g. MIR, a new intermediate layer which was needed to support an improved borrow checker (and close about a dozen soundness bugs)). Rust is happy to make the compiler implementor's life difficult in order to make the language user's life easier.

Rust also doesn't require anyone proposing RFCs to be at all familiar with the compiler internals; all that RFCs require is a loose approximation of how a given feature will change the documentation and the theoretical language reference.

And while the Rust compiler was certainly a mess before 1.0 (due to being written in an ever-changing version of itself), nowadays the compiler is much cleaner. It would also be mistaken to suggest that only a few people ever touched the compiler, as the Rust repo has, AFAICT, one of the highest contributor counts on all of Github (apparently higher than Ruby, Node, Swift, Go, Clang... so far the only ones that I've found that exceed it are rails/rails and torvalds/linux (listed, cheekily, as having "∞ contributors")).