> It's the only credible competition NVidia has
This is wrong, both AMD and Intel (through Habana) have GPUs comparable to H100s in performance.
This is wrong, both AMD and Intel (through Habana) have GPUs comparable to H100s in performance.
And AMD has ROCm. pytorch is standard and pytorch has ROCm support. And the Google TPU v5 also has pytorch support.
We do have a couple of H100's, but I'd love to replace them with AMD's
The datacenter stuff is on a different architecture and driver stack. The number one supercomputer on the top500 list (frontier at ORNL) is based on AMD GPUs and AMD is probably more invested in supporting that.
Frontier certainly has some unique quirks but the documentation is online[0] and most of these quirks are inherent to the kinds of fundamental issues you'll see on any system in the space (SLURM, etc).
However, most of the issues are fundamentally ROCm and you'll run into them on any MIxxx anywhere. I run into them frequently with supported and unsupported consumer gear all the way up.
[0] - https://docs.olcf.ornl.gov/systems/frontier_user_guide.html
I got an appointment and have exactly one minute till I have to leave, apologies for brevity: they can't open source the full driver because then they'd have to release HDMI spec stuff that the consortium says they can't. (I don't support any of that, my only intent is to communicate George isn't really locked in here when he starts casting aspersions or claiming AMD doesn't care)
CUDA is a language you write kernels. It competes with OpenAI's Triton language.
Here's what CUDA looks like: https://github.com/tspeterkim/flash-attention-minimal/blob/m...
This is what Triton looks like: https://triton-lang.org/main/getting-started/tutorials/06-fu...
By contrast Vulcan looks like this: https://github.com/KhronosGroup/Vulkan-Samples/blob/main/sam...
(It's true to some extent that maybe you could use Vulcan shaders to write deep learning kernels, maybe? I'm not aware of anyone doing it though)
If you're talking about training LLMs, involving 10's of thousands of processors, then the specifics of one processor vs another isn't the most important thing - it's the overall architecture and infrastructure in place to manage it.