Pure rust wouldn't be good for a cryptography library because it doesn't have features like constant-time functions that mitigate side channel attacks.
[1]: https://doc.rust-lang.org/1.8.0/book/inline-assembly.html
I’ve also used tooling[2] to measure any timing issues in libraries like dalek (ed25519) and it couldn’t find any (so good luck if you’re trying to exploit that over the network)
[2]: https://github.com/rozbb/dudect-bencher
Today I’d be more worried by memory safety bugs created by the use of C or assembly, or logic bugs due to only a few people being able to audit an implementation.