edit: grammar.
Additionally, this could be extended to safe code to allow removing overhead from safety in things like bounds checks and Rc.
Does it really require formal semantics for unsafe rust though? I'm not familiar enough with rust to give an example, but if you imagine there's the unsafe rust level and beneath that the "machine code" (not actually machine code, just at the abstraction level equivalent to it) you should be able to hand write what the rust code is doing, without requiring the compiler to construct the machine level operations.
With a formal semantics the proof checker just checks that the written proof (at rust level) matches with the rust code, but requires as you said an understanding of how unsafe rust interacts with the proof, whereas with a proof written at machine level, you don't need to understand the rust semantics, you just need to translate the rust to machine level and then check the proof there.
Perhaps the machine level could be some layer in llvm? I'm only a little familiar with compilers, and hardly at all with more complicated compiler theory, but this seems reasonable to me.
This approach would only be able to verify a particular compilation result as safe. If you want to verify that it will always be safe, you need to be comparing against the behavior of future compilers, which requires some kind of contract about their behavior. “Formal semantics” is the technical term for that contract.