Hacker News Reader

Comment by StilesCrisis | Hacker News Reader

I read somewhere that the expected 3-5 year lifespan of inference GPUs was not holding up to scaring: in practice, the GPUs have the same lifespan as any datacenter CPU or RAM. I think that estimate was based on Bitcoin mining ASICs, which wasn't an apples-to-apples comparison.

View on HN

Older GPUs still have an active secondary market but they are probably not profitable at market rates for inference because newer chips are so much more efficient.

If Semi-Analyses is correct [1], even H100s would lose money at current prices - I doubt that is actually true of H100s in practice because from what I can tell it is based on a 3-year depreciation from the new price, but the point is that there is a point at which old hardware's operating inefficiencies dominate.

https://inferencex.semianalysis.com/infx/profit-estimator/de...

I'm not certain it's the longevity of the actual hardware so much as there is a huge incentive to replace your hardware with faster, more capable equipment. Who wants to invest in the AI company that's using 10 year old GPUs when the next company is using brand new?

Old GPUs today are selling and renting for more than they were new which suggests the typical depreciation pattern isn’t holding.

There’s a risk that likely will happen that once demand is met capacity will overshoot significantly. However, when will that occur is very up in the air. Everyone is training every larger models and there’s more AI usage than ever

There’s still a cost to underestimating asset depreciation from an accounting perspective. Imagine being left holding the balance sheet when you budgeted for X amount of depreciation but the demand drops significantly more than you expect during the next fiscal year. I guess my point is that artificially constrained supply is not a free lunch.

Reply on news.ycombinator.com