Hacker News Reader

Comment by atonse | Hacker News Reader

I am not an ML expert but as an observer, others have said that why nvidia got right from the beginning, was actually the software support. Stuff like CUDA and good drivers and supporting libraries from over a decade ago? All the libs and researchers and all just use those libs and write software towards it. And as a result it works best on nvidia cards.

View on HN

As someone closer to this in the industry (embedded ML.. and trying to compete) I agree with the sentiment. Their software is good, I willingly admit. Porting a model to embedded is hard. With NVIDIA, you basically don’t have to port. This has paid dividends for them, pun not intended.

I don’t really see the Nvidia monopoly on ML training stopping anytime soon.

But their valuation is based on forward (future) earnings, using an already obsolete technology.

Correct me if I'm wrong, but isn't OpenAI still using a ton of Nvidia tech behind the scenes? In addition to GPUs doesn't Nvidia also have dedicated ML hardware?

Yes, and all that CUDA software is effectively a moat. ROCm exists but after getting burned badly and repeatedly by OpenCL I'm disinclined to bet on it. At best, my winnings would be avoiding the green tax, at worst, I waste months like I did on OpenCL.

That said, AMD used to be in a dire financial situation, whereas now they can afford to fix their shit and actually give chase. NVIDIA has turned the thumb screws very far and they can probably turn them considerably further before researchers jump, but far enough to justify 150x? I have doubts.

The fact that I hadn't even heard of ROCm until reading your post indicates they got a long way to go to catch up. I've heard of OpenCL but I don't know anyone who actually uses it. I think Apple has something for GPGPU for Metal with performance shaders or compute shaders, but I also don't know of anyone using it for anything, at least not in ML/AI.

It's a little irritating that Nvidia has effectively monopolized the GPGPU market so effectively; a part of me wonders if the best that that AMD could do is just make a CUDA-compatibility layer for AMD cards.

If you look at the ROCm API you'll see that it's pretty much exactly that, a CUDA compatibility layer, but an identical API means little if the functionality behind it has different quirks and caveats and that's harder to assess. I am rooting for ROCm but I can't justify betting on it myself and I suspect most of the industry is in the same boat. For now.

My question is, is it feasible for AMD to build an ahead of time compiler that transparently translates CUDA instructions into whatever AMD could have, so things Just Work™? They'd be heavily incentivized to do so. Or even put hardware CUDA translators directly into the cards?

Or am I misunderstanding CUDA? I think of it as something like OpenGL/DirectX.

In what universe is it obsolete?

In the universe of peepeepoopoo7.

Reply on news.ycombinator.com