GPU Offload in Rust: Portable, Safe, and Fast
arxiv.org
arxiv.org
I really appreciate the work and the effort that went into this. However, such an approach has previously not really worked for C++ with LLVM offload. Why would it work for Rust?
I think that will depend on the exact reason(s) C++ with LLVM offload didn't work out? If Rust differs from C++ in a way that addresses pain points/failure modes/etc. from the C++ attempt, for instance, then perhaps it isn't unreasonable to think Rust could succeed where C++ didn't (c.f., Mozilla's pre-Rust attempts to parallelize Firefox's CSS styling engine). Inversely, if Rust doesn't do things differently in the right way perhaps one might expect the effort to also not work out. Or maybe the problems are entirely non-technical and things could work out in either language.
They're very different languages, with different semantics. Without reading more than the synopsis of the paper, they're 100% leveraging the substructural type system and will have a really tight requirement for you to use a certain kind of Rust code at the CPU/GPU boundary.
I think that's why the "by default" is there; the goal is to offer a safe/convenient API that performs well enough that by default you don't need to reach beyond said safe/convenient API. And if you happen to be in a situation where the default performance of the safe/convenient APIs is insufficient, more advanced APIs will be provided.
It's a mirror of Rust's general design goals, if anything.
> It's a mirror of Rust's general design goals, if anything.
Do you claim that "it's a mirror of Rust's general design goals, if anything" to be forced into memory unsafe, extra-difficult Rust code being required, when the going already gets tough regarding performance requirements? That is not a great design goal, and it is not conducive to memory safety nor high quality software.
I mean, "by default" means "by default", not "in all cases". Situations that the default doesn't address are not inconsistent with the existence of a default; it's just that said situations are expected to be a relative minority.
Perhaps a more concrete example would help: say the API the devs come up with is sufficiently fast enough for 99 common use cases and not fast enough for 1 uncommon one. I don't think it'd be inconsistent to call said API "sufficiently fast by default" since "sufficiently fast" is an accurate description of the API for "normal" use (i.e., that's the "default" state).
> Do you claim that "it's a mirror of Rust's general design goals, if anything" to be forced into memory unsafe, extra-difficult Rust code being required, when the going already gets tough regarding performance requirements?
No, I think that's a rather... imaginative interpretation of what I said.
You needn't use your real name, of course, but for HN to be a community, users need some identity for other users to relate to. Otherwise we may as well have no usernames and no community, and that would be a different kind of forum. https://hn.algolia.com/?sort=byDate&dateRange=all&type=comme...
With C++ the last time, the problem was that there was no MLIR and it's dialects. So, the lowering could not take advantage of any vendor specific features like CUDA graphs and co-operative groups. That has apparently now change with MLIR being a possibility, but with C++, it is still in the experimental phase with CIR not finalized yet, even though people are working quiet hard on it.
With Rust, I am not sure if Rust lowers to MLIR, but if it does, then things should work otherwise not. However, knowing that all GPU vendors are working very hard on CIR and lowering to MLIR, and seeing how much time it is taking them, I would be surprised (pleasantly so) if Rust --> MLIR --> Backend would be faster.
Nvidia has started experimenting with Native Rust so it is still a possibility but still a pleasantly surprising one.
... isn't Metal shading language just C++17 compiled with LLVM ? working on every Mac and iPhone in the world is not what I would call "not really worked". Likewise, SYCL works just fine.
> just C++17 compiled with LLVM
i invite you to attempt to compile/run absolutely any C++17 codebase on your iphone's GPU lol!
If they really wanted a vendor neutral solution for Rust GPU, that already exists: you write the CPU side code, including buffering, allocation, concurrency, etc through Vulkan binding and consume the compute kernel in SPIR-V from HLSL/GLSL/WGSL etc. As it stands, the way they use Rust here feels more like using it like TypeScript types/interfaces than anything else.
Again, the size of most operations that should be done on the GPU is known ahead of time before compilation, so it's very much possible to statically allocate memory at compile time instead of going through all this trouble to write what's essentially a Rust shaped DSL for GPU compute.
But yeah, I think Python's dominance in science/ML will eventually pass, just as FORTRAN and Matlab did before.
Maybe it doesn't matter in a post AI world but perhaps it will allow better abstractions.
No need to yuck someone else's yum.
Why is Rust the only language to have literal acolytes? It's as if people believe they are part of some collective computer religion ushering in the messiah. Weird, man.
Respect for addressing something that fundamental at the design level. If something deserves praise, it’s okay to praise it. Give it a try before writing it off as religion.
Whats new is old. Ada was that breath of fresh air for me. Safety through a type system with built in concurrency since 1983. Spark gives you provable safe code.
> It’s like driving stick your whole life in heavy traffic, and someone just handed you an automatic :)
Eh, depends on the vehicle (weight, gearing, tires). My 2006 Civic was a dog in traffic but my 2002 Pathfinder was a breeze.
People also like community and rust has a decent one. It's not wrong for folks to want to feel part of a community they have shared interests or values with.
https://rustc-dev-guide.rust-lang.org/offload/internals.html https://github.com/rust-lang/rust/issues/131513
also, seems like this is mostly targeted towards HPC audience?
Technically true, since it supports NVIDIA and AMD.
But we have a different definition of portability, if I cannot bring a Metal device and expect it to work.
Why is it a blocking issue?
I feel like this is very aligned with the goals of rust-gpu.
This might be an relevant read: https://smolmachines.com/engineering/gpu-over-vsock
the connection I'm making is that both sit on the same interface: the CUDA driver api (module load, launch, memcpy).
the paper is about compiling kernels using safe rust instead of CUDA C++, what I did is reimplement the other side of it using rust for the calls from inside a VM get forwarded over vsock to the host driver via api-remoting.
Today I learn through GPU MODE & their discord: https://www.gpumode.com/events
I've only built toy kernels to experiment with supporting remoting CUDA workloads on smolvm: https://github.com/smol-machines/smolvm/blob/6dcb182d7fbc9f5...