Going above 24GB is probably not going to be cheap until gddr7 is out, and even that will only push it to 36gb. The fancier stacked gddr6 stuff is probably pretty expensive and you can’t just add more dies because of signal integrity issues.
Which I don't care too much about.
However, even 16->24GB is a big step, since a lot of the model are developed for 3090/4090-class hardware. 36GB would place it lose to the class of the fancy 40GB data center cards.
If Intel decided to push VRAM, it will definitely have a market. Critically, a lot of folks will also be incentivized to make software compatible, since it will be the cheapest way to run models.
I want a consumer card that can do some number of tokens per second. I do not need a monster that can serve as the basis for a startup.
The hobbyist market needs something priced well under $1k to make it accessible.
If a model takes twice as long to run.... I'll live. Worst-case, it will be mildly annoying.
If I can't run a model, that's a critical failure.
There's a huge step up CPU->GPU which I need, but 3060 versus 4090 isn't a big deal at all. Indeed, the 24GB versus 16GB is a bigger difference than the number of CUDA cores.