Your assumption is that memory safety has to be baked in the language. It could be baked into proof assistants that are part of (optional or add-on) tooling, like what sel4 does. A simpler language makes this more possible, and the things that a proof assistants can do go far beyond what rust is able to provide, without sacrificing compilation speed or other forms of optimality (e.g. avoiding the heap) when you don't want or need such a high level of security guarantee (e.g. writing a cli tool that never sees the internet)
As for me, I'm terrified that rusts complex macro system will hide/obfuscate discovery of other forms of security regression, like timing or energy side channels.