Given the current situation, this is wishful thinking/a long way off. Nvidia is considerably better at letting people get the job done on consumer grade GPUs than AMD.
Tonight I'm going to play around with LLMs. I'm probably going to go to bed early because my AMD graphics card running ROCm (on an unsupported system and a probably unsupported graphics card) will eventually cause a kernel panic.
In some sense that is unforgivable. In other senses, they are only 1-2 bugs away from being perfectly good enough and competitive with Nvidia for me. That isn't much of a moat. It usually takes an hour or so for the drivers to collapse and for those hours everything works great.
AMD also said this 24 months ago.
We'll see how smart the market and the monopolist are this time around, yes, in the next 12-24 months... and maybe longer.
I was specifically referring to AMD's investments in a unified computing stack (their equivalent of CUDA) which were supposed to bring them to computational parity in the AI/ML.
https://www.extremetech.com/computing/here-we-go-again-ai-de...
Retail gaming GPUs are available and they’re MUCH cheaper than cloud GPU.
It’s happening now.
AI software needs to make a choice too, does it only run in Nvidia, or does it make itself compatible with the mass market.
For 50 years software developers have chosen to make their software work on cheap mass market devices.
However, some ML models (such as LLMs) demand a lot of vram to train. Simultaneously, a lot of gamers have been unimpressed by the latest generations of gaming GPUs due to the limited vram.
nvidia's GTX-1070 released in 2016 with 8 GB of vram and an MSRP of $379. Then the RTX-2060 Super, with 8 GB of vram and an MSRP of $399. Then the RTX-3060 Ti, with 8 GB of vram and an MSRP of $399. Then the RTX-4060 Ti with - you guessed it - 8 GB of vram and an MSRP of $399.
Some people think nvidia is deliberately being miserly with vram on gaming cards, to try and force ML users onto the $$$$$ data centre GPUs.
I suspect this is what andrewstuart means by "Nvidia is pushing upmarket" - that they're deliberately letting their gaming products languish, in pursuit of the data centre market.
4GB (from personal experience, 2GB from what I’ve heard) is enough to run stable diffusion 1.x/2.x with the major current consumer UIs and the optimizations they make under the hood. Not sure about SDXL. The original first-party inferencing code takes more.
> Some people think nvidia is deliberately being miserly with vram on gaming cards, to try and force ML users onto the $$$$$ data centre GPUs.
And, sure, as long as there is no real competition, that kind fo segmentation makes sense. OTOH, if there is competition and desktop ML demand, it will make progressively less sense. So, the question seems to be, will there be competition that matters?
Who's making that choice? It sounds like a lot of people want to make this stuff run well on iPhone and Android and AMD and Mac, but they just don't have the API control or insider help. This isn't a thing where Open Source developers step up to the task and fix everything with a magic wand. This is a situation where the entire industry converges on a compute standard, or Nvidia continues to dominate. They are betting on Apple, AMD and Intel being stuck in an eternal grudge-match, and their residuals won't stop rolling in.
The real question is much less hopeful; can the industry put aside it's differences to make a net positive experience for consumers?
...probably not. None of the successful companies do, anyways.
I say this as a person who's diving right into the NVidia moat. "Well everyone gets NVIdia GPUs so that's what I should get".
this isn't in the hands of the industry as a whole, though. The question is: Does AMD think they can earn a better profit down the road than if they invest in/focus on something else (e.g. maintaining their lead over intel on x86, continue being dominant in the console gaming GPU sector, etc...).
So what? “The industry as a whole” isn’t an actor that makes decisions.