Hacker News Reader

Comment by dragonwriter | Hacker News Reader

> This is just temporary though, right?

Well, everything—even human life on Earth—is just temporary, but RAM supply lagging centralized-AI-driven demand increases continuing to squeeze the consumer market may not be a short term phenomenon.

> LLMs seems quite simple, maybe we'll be able to print/assemble at home our own chips with the desired models/weights.

So, the solution to the RAM crunch is “everyone has their own home chip fab and deals with the raw material supply and hazardous waste disposal”?

I...don't imagine so.

View on HN

Hazardous waste?

My idea was more like you get some pre-made chips, that you can maybe assemble together configure at home with your desired models.

Maybe each one of this chip is a layer, so you can stack as many layers as you want.

HBM modules that we can just attach to our GPU? Kinda like CPU RAM.

You're not thinking through the way economics works.

If, hypothetically, the thing you wish existed, why wouldn't it be valuable to high-utilization cloud providers?

And if useful to them, why would it be sold at a lower price than the (extremely high) price those cloud providers can afford to profitably pay?

The only solution to the hardware crunch that eases the cost/access barrier is if there's such a glut of excess supply of newly-commoditized inference hardware that the price craters.

And you've yet to propose a feasible manufacturing scenario through which that would be possible.

Well, before AI the RAM prices were cheapest ever, even if cloud providers were still in demand for hosting and cloud computing.

I hope the supply will increase at some point, I doubt cloud providers will be able to buy everything, especially if the competition will be high and the price of AI goes down, so there won't be an easy way to make profits by serving models.

> so there won't be an easy way to make profits by serving models

That's the only way I see things going, which ultimately comes down to the market losing faith in AI as a profit engine.

Which is probably going to take one of (a) OpenAI / Anthropic IPO failing or newly public financials looking bleak or (b) a high-enough profile large AI company failing (unfortunately most of the large enough to shock the market ones have diversified and durable cash generation engines).

Unfortunately?

So the solution to the chip supply problem is “pretend there is no chip supply problem and everyone gets the chips they need, but for some reason does some assembly task at home”?

If we could just do the first part and get reality to conform, the rest would be superfluous, but I don't think we can do that.

This is not a solution for now, just a moonshot solution for maybe 5-10 years from now.

There's no immediate fix for the RAM supply issue, but as long as it's manufacturing issue, not a raw materials issue, time always solves it.

Its not a solution ever, because it takes as its premise that the problem is solved and then goes on to suggest something else on top of it (the purpose of which is unclear.)

Reply on news.ycombinator.com