Hacker News Reader

Comment by Havoc | Hacker News Reader

The bigger issue I suspect is the ram.

Don’t think even DDR5 gets anywhere close to gddr6x on dedicated GPUs. Plus gpu side ram tech is due for a refresh soon too - it’s been on gddr6x for a while

View on HN

The flipside of that is capacity though - consumer GPUs cap out at 24GB, while consumer CPUs can now support up to 192GB, and the delta is similar for professional/enterprise hardware. If iGPUs are scaled up to higher performance tiers then they could have a niche in crunching very large datasets that would have to go out-of-core on a dedicated GPU.

I'm hoping that Intel's integration of HBM on CPUs trickles down into something that doesn't cost $10k. It puts CPU memory bandwidth in the same ballpark as GPUs. And honestly at 16-32GB, it's enough for some whole systems, without needing DDR5 DIMMs.

Not on consumer stuff. It is just too expensive per GB, even if you already have the interposer for something else.

LPDDR5X on a wide bus (and a healthy LLC) is fine, it just needs implementations beyond Apple.

With a wide enough memory bus, it’d be possible to make a CPU and a GPU happy at the same time with the same memory system, but we’d need manage their expectations about how much they can hog the bus.

Low-end cards have historically had lower-end GRAM.

Plus, graphics-oriented RAM is just optimized differently: for bandwidth at the expense of latency.

GDDR trades bandwidth for latency. Modern GDDR bandwidth is one order of magnitude above modern DDR. And latency is much worse.

Traditional low end GPUs used regular DDR3 or DDR2 for cheapness.

The low end GPU market has become non-existent today. At a minimum, every GPU, even the cheapest, has large amounts of specialized high bandwidth RAM.

OTOH, there might be a niche for a GPU that has a lower bandwidth access to a larger pool of memory, because inference might consume a lot of data and not that much number crunching.

That's called a CPU with AVX512. Modern EPYCs can give 2TFlops or more while physically connected to 2TB of RAM.

Would be if we could have many hundreds of cores doing AVX512.

We can, on the very high end.

Reply on news.ycombinator.com