Hacker News Reader

Comment by cmiles8 | Hacker News Reader

Except:

1. GPUs are reportedly increasingly just getting put on shelves because there aren’t data centers to host them, which is creating a setup for a financial mess (rapidly depreciating assets purchased, paid for, but not utilized),

2. And massive chunk of the compute “demand” is coming from circular financing (ie AWS is “sold out” in large part because it gave startups like Anthropic money to buy the thing it’s now telling the market is “sold out” so there’s some shenanigans behind those headlines.)

I do agree though that compute is really the only place to make money in AI in the long run. Models are an increasingly a commodity with little value above the cost of compute to run them. However selling compute will be with razor thin margins, not the fat AWS margins the market got used to in cloud, so AWS and others will have to deal with that uncomfortable reality.

View on HN

I worry they’re going to destroy the chips they can’t afford to deploy to keep them from being used for local inference

How would chips that, say, Amazon paid for end up in local devices?

I don’t understand the scenario that destroying the chips would guard against. Amazon could just leave these chips in a warehouse indefinitely.

I guess this is functionally destroying them? Doesnt take long for chips to go out of date. If I buy a load of potatoes and let them rot, have I not destroyed them?

> If I buy a load of potatoes and let them rot, have I not destroyed them?

Depends on how they're stored. Computer chips aren't potato chips, they don't rot, properly stored they can last a really long time, a multitude of storage technologies are available for that purpose.

> Doesnt take long for chips to go out of date.

That used to be the case but it's no longer true. Advanced chips are made by 3 companies total and competition is nowhere to be seen. Then, with every generation of chips, their production costs and prices are rising more than the benefits that generation provides i.e. the benefits curve is flattening faster than cost curve.

Theoretically, China can bring the prices down but they are sanctioned, tariffed and have a huge market of their own to satisfy first.

> 1. GPUs are reportedly increasingly just getting put on shelves because there aren’t data centers to host them, which is creating a setup for a financial mess (rapidly depreciating assets purchased, paid for, but not utilized),

Unfortunately, "paid for, but not utilized" doesn't result in rapid depreciation in the case of coordinated market scalping. In that case, dark (unused) hardware is kept on storage to actually cause higher prices until them chips can be sold for profit, in the form of services, after eventual deployment.

I suspect, digging a bit deeper would find some shady financial practices that may lead to a taxpayer funded bailout via the insurance loophole.

> rapidly depreciating assets purchased

GPU's produced in 2020, with a supposedly "3 year life before depreciating to zero" are now sold at twice their price from 2020.

So did houses in 2007. That’s not a good sign at all.

I guess GPUs are interesting because unlike houses, we've suddenly found a massive amount of new utility for these things. If my house suddenly started producing its own furniture, it might be a bit more valuable. Which isn't to say people haven't found new things to do with their houses - COVID made having a home office a lot more valuable, for example, but nothing quite as dramatic as GPUs suddenly able to produce their own code.

In 2008 we had both a massive under supply of housing in many cities plus crashing prices. These things can very much exist at the same time if, as if happening now with GPUs, the financial engine fueling the ecosystem is out of whack.

Reply on news.ycombinator.com