At least with the more common modern ML (LLMs etc) operations, no one drops down to Rust. It's really only good for IO/CPU work, but that's rarely a spot for new operations, just for data loading and high-latency network stuff.

People typically use Triton or CUDA (usually the C++ flavor) to implement operations. It'd be cool if Rust had a CUDA dialect.