Hacker News Reader

Comment by tmaly | Hacker News Reader

Unless Sam gets his 7T I can’t see the data center growth continuing on pace.

I still would love to see a consumer card with 128GB of VRAM

I would love to see a rethinking of desktop hardware. GPU's are huge. They used to be dinky little add-on cards ... but at this point we almost need to have a dedicated mainboard for them.

Imagine this - instead of a motherboard mounted to the back of the case, it is mounted in the center. Behind it would be another motherboard, with a socket for a GPU and ram slots for memory. Kinda like a right/left brain connected with a high-speed interconnect. You open the left side of the case to access CPU and its memory, you open the right side of the case to access the GPU and its memory.

You would be able to upgrade your GPU chip independently of everything else. It wouldn't be mounted sideways in a PCI slot where additional brackets and support arms are needed. You could upgrade the RAM independently. Cooler, etc.

What you're describing is fairly close to Intel's Beast Canion NUC [0] from a couple of years back. The Mobo and GFX essentially sit back to back. Though intel seemed to envisage upgrading the whole "CPU side" as a single piece (including ram) which probably, along with its high price, put off buyers.

Following on from that though, there are a number of miniitx cases that followed on from that type of layout such as the fractal terra [1] which utilize a pcie riser to pass it rearward to the graphics card behind the motherboard. ATX/ITX wasn't entirely designed for it, but the layout does seem to be quite elegant.

[0] https://www.tomshardware.com/reviews/nuc-11-extreme-kit-beas...

[1] https://www.fractal-design.com/products/cases/terra/terra/te...

There is no consumer use case for that so basically inexistent market. It would be rather dumb of Nvidia (or any other for that matter) to willingly cut its entreprise margins for such a non product.

The only one doing lots of VRAM are Apple and nobody really care because performance is not that competitive and price is about as bad as entreprise stuff.

I still think modular arch will stay for consumers products there much more benefits to that than chasing a bit more perf for low volume products. Apple can get away wit their shenanigans because they subsidize their high-end chips with the iPhone. If they could only sell Macs it would extremely unsustainable (like the PowerPC venture showed).

I would also love to see a consumer GPU with a lot of VRAM. I think 128 GB would be too expensive BUT I'm hoping the Nvidia 50 series comes with 32+ GB VRAM for at least the top tier cards of the family (today they max out at 24 GB VRAM in the 3090 and 4090.)

imagine how impossibly sold out those cards would be.

4090s are only sold out because of the China sanctions and are moving back towards MSRP. Fundamentally even a 48GB desktop card would be no substitute for a B100.

yeah but if you had 4090s with H100 equiv VRAM (prob don't even need HBM equiv) that wouldn't require quantizing larger models you would end up with a ton of LLM workstations at a fraction of what they charge for H100s. The demand would be high. What China was doing with the 4090s requires a great deal of technical/manufacturing prowess that doesn't apply to most.

There is one. Get Apple M3 with 128GB ram :)

based on what I've seen from IDC and Gartner in fact it will keep growing for another 2-3 years in insane fashion at least.

Reply on news.ycombinator.com