Of course, you might then ask "what's the point of using Rust at all?" The answer is that a lot of code can be expressed in regular patterns idiomatic to the Rust type system and borrow checker. This isn't particularly different from any other type of programming. You sometimes see people say "Rust is useless because you can't safely write an arbitrary graph with direct pointers". The obvious fallacy with this argument is that most code doesn't need an arbitrary graph with direct pointers; for the small amount of code that does, you can drop into unsafe in a targeted way and still have a much safer program than one written in e.g. C. GPU programming is no different.
[1] https://www.cl.cam.ac.uk/~pes20/cpp/cpp0xmappings.html
(Note than on x86 Relaxed atomics turn into plain mov instructions, and on ARM they turn into plain ldr and str.)