AMD is not too far behind. They can go wild in a few years.
AMD hasn't done anything substantive in the past two years since GPT-3/GPT-3.5. Almost every research paper implements their algorithm in CUDA. AMD can't beat software with good hardware.
I'm at a startup, and we'd love to be using MI300Xs. Out stack works with it, they are amazing, our wallet is open... But we can't! We simply can't find any. It seems they are unobtanium for megacaps only.
Nvidia is in the limelight, but their product (GPU compute) is a commodity.
Once someone else has it cheaper, then it’s a race to the bottom.
The recent news was about AMD giving up on that path.
They still funded it and it was created.
They gave it to us instead of tossing it into the trash.
I don't know what you mean to imply by that.
Just because they don’t want it doesn’t mean it just vanishes or stops working.
> Shortly thereafter I got in contact with AMD and in early 2022 I have left Intel and signed a ZLUDA development contract with AMD. Once again I was asked for a far-reaching discretion: not to advertise the fact that AMD is evaluating ZLUDA and definitely not to make any commits to the public ZLUDA repo. After two years of development and some deliberation, AMD decided that there is no business case for running CUDA applications on AMD GPUs. > > One of the terms of my contract with AMD was that if AMD did not find it fit for further development, I could release it. Which brings us to today.
It's worth noting that while ZLUDA is a very cool project, it's probably not so relevant for ML. Also from the README:
> PyTorch received very little testing. ZLUDA's coverage of cuDNN APIs is very minimal (just enough to run ResNet-50) and realistically you won't get much running. > However if you are interested in trying it out you need to build it from sources with the settings below. Default PyTorch does not ship PTX and uses bundled NCCL which also builds without PTX:
PyTorch has OOTB ROCm support btw and while there are some CUDA-only libraries I'd like (FA2 for RDNA, bitsandbytes, ctranslate2, FlashInfer among others), I think sponsoring direct porting/upstreaming compatibility of the libraries probably makes more sense. Also from the ZLUDA README:
> ZLUDA offers limited support for performance libraries (cuDNN, cuBLAS, cuSPARSE, cuFFT, OptiX, NCCL).