Yes, but Rust might still be able to do what other languages can't. For example, to express a given program using at most N input symbols.
(Actually, upon further thought, it may be that rustc rejects all invalid Rust programs and some valid ones. gcc certainly accepts some invalid C programs.)