Hacker News Reader

Comment by AnimalMuppet | Hacker News Reader

OK, but Altman is complaining about individual users. Nobody said that maxing out my PC for gaming purposes or just to keep Windows from getting sluggish was a waste, pretty much ever.

And, "waste" has an overtone of moral wrong, of "you shouldn't". When before did anyone say that I shouldn't get a nice PC? Never.

So, yeah, moralizing to individual users that they shouldn't buy DRAM so someone else can have it is new.

View on HN

He doesn't say anything about individual users. It's left unclear exactly what he means when he talks about low utilization inference using DRAM. I figured he was talking about companies building clusters. I see way more people using old GPUs for local inference than buying 15-20k Mac Studios, or RTX Sparks. I've seen plenty of stories about how companies are building clusters because they want to run open source models, partly for privacy purposes, partly because they're afraid of a rug pull, partly because they want to control when they move to new models, etc, etc.

> He doesn't say anything about individual users. It's left unclear exactly what he means when he talks about low utilization inference using DRAM. I figured he was talking about companies building clusters.

No, he absolutely does:

"From a global allocation perspective, actually, it's quite terrible to have everyone have a bunch of the world's limited supply of DRAM sitting in their own computer. And be used 5% of the time," Altman said.

Reply on news.ycombinator.com