I'm genuinely rooting for AMD to develop a competitive alternative to NVIDIA. Currently, NVIDIA dominates the machine learning landscape, and there doesn't seem to be a justifiable reason for the price discrepancy between the RTX 4090 and the A100.
At the same time, the cost is outrageous. Its not a low volume product.
Also, a 48GB 4090 (or 3090) would be trivial. So would a 48GB 7900. Its not done for purely anticompetitive reasons, one that AMD and Nvidia are unfortunately happy to go along with.
48Gb 3090: https://www.nvidia.com/en-us/design-visualization/rtx-a6000/
https://www.bhphotovideo.com/c/product/1753962-REG/pny_vcnrt...
As most economics 101 lecture would tell you, price is determined by supply and demand and in this case Nvidia essentially maximizing their profit based on the demand and segmentation of the market.
Is this why Intel started taking dGPU production more seriously in recent years?
Nvidia and AMD used to "let" their manufacturers double up the vram on gaming cards, but no more. Its pure, collusive, anticompetitive market segmentation.
https://github.com/RadeonOpenCompute/ROCm/issues/2198#issuec...
One might speculate he’s perhaps pivoting to Intel. They’re not well-developed in application terms but that’s a piece he can develop, and actually with OneAPI that’s a lot of potential bang for the buck. And intel has actual ML accelerators and has some relatively powerful GPGPU stuff and is currently in a position of being forced to offer a lot of bang for buck to drive adoption, all of which makes sense for what he’s trying to do.
But AMD wants you to basically write and debug their runtime for them and nah not worth it, after fighting the installer on the official system config and then filing a couple bugs for the demo apps reproducibly crashing the kernel it’s just not worth the time.
ROCm is unserious even when you’re operating on supported hardware. This is the experience most people have with it.
You know, it doesn't surprise me that's what they're doing, because the HPC people aren't paying megabucks and then tolerating something that crashes the kernel... but people's problems/experiences aren't invalid either, and the delta is they're not running the same software. I don't like it but it makes sense.
1. they're a publicly listed company operating in a free market in an industry they themselves helped develop who's purpose is to make returns for their investors, not be liked by the public, they don't need to justify their pricing to their buyers. They're not selling essentials for survival like insulin, baby formula, or housing, they can charge as much as the market will bear for their consumer electronics products. Don't like the pricing? Don't buy it. Simple. Buy from the competition instead or older generations off the second hand market that fit your budget.
2. the price justification is that cutting edge silicon is and will always be in short supply, and buyers of the silicon in A100 form, like datacenters, use it to make money, therefore it's an investment that will yield returns, therefore they can justify spending way more money to outbid the gamers who buy the same silicon in RTX 4090 form and don't use it to make money but use it to play games therefore for them the product is worth less and it makes Nvidia smaller margins than what selling it to datacenters can. It's basic price segmentation that's been going on for decades.
I also don't like the GPU pricing situation but that's the market reality I can't change and downvoting the messenger won't change it either. My 2 cents.
AMD has been consistently making bad business decisions in the GPU space for years now. What's new? They're pricing their hardware near NVidia but with less performance and way less features and poor track record, especially for the ML users.
Thinking that Nvidia is some evil villain doing it to consumer out of spite, when they're just doing what any other company in their dominant position would do: charge as much as the market will bear.
Apple also doesn't have to charge you $200 for configuring the 512GB SSD over the 256 SSD which only costs them an extra $5 NAND chip, but they do it because they can. So does Nvidia and any other company in a dominant position with virtually no competition.
You might be reading into their comment too much, they don't really blame NVidia, just wish for more competition. IMHO you're being downvoted because you come across as needlessly confrontational to an innocent comment.
Maybe I'm wrong and OP really didn't understand the curent supply/demand issues which is why I gave a lengthy explanation of how it looks.
LLms running on standalone affordable boxes could become a ubiquitous home accessory, not to mention the potential for things like being able to leverage learning from widespread use to retrain the models, like Tesla does with their fleet.
Yes, exactly like all the big tech companies with vast departments for market research that hired a shitload of people, then when economy turned, they started doing layoffs. A.k.a short term profit seeking.
It's a dick move but it's not illegal (at least not now). Their purpose is to make greater returns for their investors, not make affordable ML hardware for every consumer.
>LLms running on standalone affordable boxes could become a ubiquitous home accessory
They could be, but why is it Nvidia's fault if they don't happen? They're in the business of making money, not fulfilling dreams, and currently they can barely fulfill their orders for the datacenter customers. Consumers could move the LLM stuff on cheaper Apple HW or Intel or AMD SoCs if they can't outbid the datacenter companies for Nvidia silicone.
Seems like a market opening for Nvidia's competitors to price cut them. If they don't exploit it and let Nvidia dominate, it's their fault. It's not Nvidia's fault their competitors were asleep at the wheel since they launched CUDA in 2007 and were helping researchers put their GPUs to use for parallel computing for over 15 years.
Something like chrome : chromium dynamic would work wonders for bringing a sort of standard, not to mention more minds could create better and better implementations.
By that time nvidia already had an advantage with CUDA and their opencl support always lagged.