Hacker News Reader

Comment by adityaathalye | Hacker News Reader

> The reason why AI companies are spending so much money is to rollout data centres to run the AI in the first place. These expenses are one-off purchases, not regular operating expenses. Once all the data centres are built, AI spending is going to plummet.

Fatal misunderstanding.

Datacenters are not warehouses. Amazon.com being borderline-profitable owing to massive ongoing capex on warehouse expansion is not at all like AWS becoming borderline-profitable owing to massive ongoing capex on LLM datacenter build-out and SOTA model training (it's CAPEX, but carries OPEX affect because of GPU depreciation schedules and competitive hardware upgrade cycles), and opex of delivering LLM inference services, and being able to price it all competitively.

(edit: expand explanation of why it's a misunderstanding)

View on HN

There's also a failure to misunderstand the arms race nature of it all and the need to be in the lead since everyone is assuming it's a winner take all market. That's probably close to correct.

A data center is a warehouse for sure but the building is only a few percent of the cost. The GPUs are probably 80% of the total cost or more. And they probably only have a 3-5 year primary lifecycle.

Pay the building off over 20 years? Sure but 80% of your capex problems remain. 2 trillion already spent? Just those GPUs need to generate 500 billion a year over the next 5 years.

Power generation is also extremely expensive ongoing - either through gas turbines that eat fuel or on-grid power that datacenters will need to pay to capitalize (depending on ratepayer agreements of course - but clearly people aren’t stoked about subsidizing these)

In aggregate it’s a lot of electricity, enough to impact local grid markets.

But as a fraction of inference revenue, there is good reason to estimate that the direct electricity costs are perhaps 1-5% of inference revenue.

That’s a lot, but hypothetically if electricity prices double, it’s more likely to lead to angry residential consumers than a doubling of inference token prices.

Lawrence Berkeley National Lab, November 2026: https://eta-publications.lbl.gov/sites/default/files/2026-11...

There are substantial economies of scale on both power generation (grid-scale) and inference power consumption (batching and utilization).

> That’s a lot, but hypothetically if electricity prices double, it’s more likely to lead to angry residential consumers than a doubling of inference token prices.

But that’s a feedback loop that could lead to more supply being built. Or it could lead to regulations that put enormous surcharges and low limits on data center power consumers, which effectively shield consumers directly from the lack of supply. Which in turn removes the incentive for consumers to expand power generation capacity to keep up (or to allow data centers to expand it themselves).

There is enormous opposition to a new data center project 30 miles outside my city because they want to build a 1000MW natural gas power plant and a 500MW biomass plant (that will take millions of acres of forest to sustain).

If that opposition continues, I see little chance of anything but electricity cost rising as a percent of inference.

Normally you'd replace the device because there is more power efficient one on the market so running one at current energy prices costs more than buying new, more power efficient one and saving power.

IF prices of GPU/RAM come back to pre bubble costs, we might see the 5 year cycle of replacement that's relatively common in normal datacenters pre-ai bubble.

At current prices ? those GPUs that are there now will be running there in 10 years still.

> At current prices ? those GPUs that are there now will be running there in 10 years still.

If they don't burn out after two or three. Annualized failure rates are about 9% according to Meta.

A quarter will have failed within three years. Less than half of the original hardware will be running after ten years. In both cases if you assume that rate does not increase over time. Do H100s come with replacement guarantees?

https://arxiv.org/pdf/2503.11901

This paper puts the failure rate of A100s closer to 0.3%/yr, H100s under 1%.

I guess we will see. But I would not expect GPU hardware to last that long until there is some underutilisation in data centres.

Mind you, most of the H100s that have ever been sold are still in their packaging, I suspect. Or at least not plugged in.

Yep. Satya Nadella once quipped that they have GPUs sitting on shelves because they don't have enough power.

> [...] because quite frankly, the biggest issue we are now having is not a compute glut, but it’s power — it’s sort of the ability to get the builds done fast enough close to power. So, if you can’t do that, you may actually have a bunch of chips sitting in inventory that I can’t plug in. In fact, that is my problem today. It’s not a supply issue of chips; it’s actually the fact that I don’t have warm shells to plug into.

https://www.tomshardware.com/tech-industry/artificial-intell...

I'm of two minds on this issue. Off the cuff I agree, but then again where are all the used "enterprise" GPUs coming from right now? There's plenty of secondary market for used Telas K, M and V series GPUs right now and the V100 is only 9ish years old, and that only if you bought it on the release day.

I do agree that 5 years isn't the hard cutoff I was suggesting but I'm not convinced that 10 years is a reasonable timeframe to consider paying the debt down when Moore's Law is involved. I know it's no longer the 12-18mo doubling it used to be but even if it's 2ish years then over 6 years you get 3 doublings or 8x performance gains. At 10 years it's another 2 doublings or 32x performance gains. It's hard for linear payback curves to deal with exponential performance curves past 2-3 doublings.

> everyone is assuming it's a winner take all market.

I guess this is the big gamble, the idea that if you just spend enough money as fast as you can you can become the gods of AI forever and can just charge whatever you want for your magic beans.

But what is the moat? If in reality these things become commodities and just compete with each other to spit out cheap tokens then there's no way the margins are there.

> and can just charge whatever you want for your magic beans.

'Winner take all' doesn't mean convincing others to buy your product. If others have something to buy your product with, you've not taken all, fundamentally.

'Winner take all' means having the most powerful army. The assumption is that once AI crosses a certain threshold then there won't be anything the rest of us can do to stop the machine of whomever gets there first, so they will truly be able to take all. Hence why the AI labs sometimes hint about the risk of human destruction.

The army interpretation exists, and might sadly be relevant here, but it's not the only one. Google, for instance, won a nonviolent winner-take-all competition for search.

You're right that Google has been greatly helped by having what has been considered the most powerful army (the US army) backing them. They wouldn't have had a hope in hell of surpassing the "mom and pop" stage without that.

But I am not sure you are right that it is powerful enough to secure all. I search daily, but cannot remember the last time I used Google. The other AI companies do it better. And I'm not alone in that sentiment. Google's stock price recently took a nosedive because investors have grown quite worried about the competition. Without a sufficiently powerful army stopping them, there will always be someone else ready to try to outdo you.

Different setup. IIRC, Google had fundamentally better tech than its competitors and was also the first to figure out search monetization.

With AI, it looks to me like competition is much tighter, with no company having fundamentally superior technology or monetization plan. So a winner-take-all scenario is much less likely.

If Anthropic or OpenAI will take all using the most powerful army why they would give anything back to their investors? If that's a possibility than investing into AI companies is just gifting money to whoever will control this army.

Anthropic/OpenAI and the investors are the same thing. A company name is just a title for the people who own it.

Maybe you mean that you could see the employees of those companies (although I imagine they also own it, so that's a wrinkle) staging a coup to seize the army for themselves? That's certainly possible. They'd be in a good position to do so. Predicting the future is impossible, but if you want to be on the "right side" of that future army, owning its foundation today is still a reasonable bet.

> everyone is assuming it's a winner take all market

I don't think anyone serious believes this any more. The tech is thoroughly commoditized. There are a lot of plays to be made, like OpenAI cornering memory, or spending heavily on unique data for RL, if you can stop distilling, but there is no proprietary core IP at all.

There is no way to monopolize a technology this big. It would be like monopolizing literacy.

It's not the AI technology itself that is being monopolized, it's the data necessary to train it.

More specifically: when everything you could previously have earned money with is subsumed by AI, the "frontier" of profitable knowledge exists solely with the highest concentration of capital. Because that's where the highest amount of compute and consequently valuable data is.

Already now the idea of the tech being "commoditized" is true only if you exclusively look at the bubble of rich countries. AI-worthy compute is prohibitively expensive otherwise for individuals.

Specifically with regards to the data necessary to train, the AI companies have allowed the earth to be scorched (to say scorched earth tactic is to attribute intent, which I wish to avoid for my point), making clean training data harder to obtain. Essentially, what I do see as intended is the enclosure of an intellectual commons.

Power, natural gas, gasoline, water, mobile contracts, railways, internet access are probably better analogies. They have tendencies to (local) monopolies but also a lot of precedent and knowledge on how to regulate and break them, how to introduce competition.

they broke up Ma Bell, but they've never done anything about the internet access. Mobile is an oligopoly.

The breakup of AT&T was the proximal cause of Ronald Reagan and Robert Bork installing the CWS shackles onto the US antitrust apparatus.

Here's the thing: we could remove those shackles. US antitrust law had a good 50 year run, it could have another. Lina Khan was working on it, you probably saw the flak. This was on the ballot in 2024 and it will be again in 2028.

I don't think so. You can't practically buy electricity from halfway across the planet but you can buy tokens. There's no local infrastructure to monopolise.

The moat was "AI", as in, self-improving Superintelligence (Nick Bostrom-style). They thought they were _just_ on the cusp of this happening, thus hundreds of billions of dollars was small fry. At that point either your AI figures out how to make money, or money doesn't even matter.

A "failure to misunderstand" is a fitting turn of phrase.

People not only ignore second order effects, they also don't think about what would actually happen when a "winner" really "took all".

While AI is a tremendously useful technology, business types wildly misunderstand it and its consequences. Greed eats brain.

You got me on that one, geez! I'm not going to edit it though because I think you're right. Nobody really knows if there's going to be a winner-take-all or not. During the arms-race portion though people are definitely acting like it will be that way. But watching how fast models catch up to each other it's not obvious it will play out that way.

> And they probably only have a 3-5 year primary lifecycle.

I think in 3-5 years close to 100% of current GPUs/RAM will still be in service. New hardware matters only if gains in efficiency are large enough to put you out of business because the competition gets them. Extreme case of that are bitcoin miners. With AI inference there are way too many factors that influence the profitability. It's not like you can't run new model without the new hardware. It can just run on more of the old one. 3-5 years for GPU are estimates when the main application was gaming with new games with new requirements for running on a single GPU being constantly released.

The problem is likely the GPU price, NVidia has a very large margin - but it doesn't seem like the the use case supports it. If NVidia cut the GPU price by 2-4x then these problems likely go away.

I wouldn't be surprised if most hyperscalers are betting on cheaper GPUs on their next upgrade through either pricing pressure on NVidia, competition from AMD/Broadcom or in-house chips.

It's probably closer to 45-60%.

https://epoch.ai/data-insights/ai-datacenter-cost-breakdown

https://www.csis.org/analysis/impact-tariffs-ai-data-center-...

Thanks for the links!

The first one says that networking infrastructure costs 13% and the other says 1.5%. Not a great start.

Let's imagine that a traditional full rack can host 80ish H100s and that each one costs $25k. That's 2 million right there. There's probably only 10x cases in that rack so that's 10x dual core motherboards, RAM, network, etc. Call them 20k each. Maybe you need a 32 port 100gig fiber switch, that's 8k. Let's say it's 2.3 million per rack and 2 million of it is just the GPUs. We're at 85% of the rack cost right there in GPU, but the other stuff in the rack isn't durable either.

I suspect that it would take more than just another 100k per rack to finish out a data center so my "80% of the cost is in the GPUs" is definitely wrong. But I didn't see anything in these articles that indicated to me that 50% of the cost of a datacenter is anything really durable like buildings or electrical infrastructure or cooling. One rack might only take up 20ish sqft itself another 100sqft of isle. These buildings don't cost $10k/sqft to build the envelope. There's still an enormous amount of cost in stuff whose performance follows more of an exponential performance curve, not linear. It's just an absolute nightmare to try and amortize exponential performance curves with linear repayment.

These days there's usually two networks. Multi-trillion parameter models are too big to fit on one box of 8x H100's, so many of these are running RDMA (800Gb/sec on GB300 series) for rack-scale/many row-scale inference.

For GB300 series, pricing along the lines of $500k/server, 9 servers/rack ($4.5M/rack) before looking at water cooling, 800Gb RDMA network, and your 100Gb IP network

I read somewhere that the expected 3-5 year lifespan of inference GPUs was not holding up to scaring: in practice, the GPUs have the same lifespan as any datacenter CPU or RAM. I think that estimate was based on Bitcoin mining ASICs, which wasn't an apples-to-apples comparison.

I'm not certain it's the longevity of the actual hardware so much as there is a huge incentive to replace your hardware with faster, more capable equipment. Who wants to invest in the AI company that's using 10 year old GPUs when the next company is using brand new?

Old GPUs today are selling and renting for more than they were new which suggests the typical depreciation pattern isn’t holding.

There’s a risk that likely will happen that once demand is met capacity will overshoot significantly. However, when will that occur is very up in the air. Everyone is training every larger models and there’s more AI usage than ever

There’s still a cost to underestimating asset depreciation from an accounting perspective. Imagine being left holding the balance sheet when you budgeted for X amount of depreciation but the demand drops significantly more than you expect during the next fiscal year. I guess my point is that artificially constrained supply is not a free lunch.

Older GPUs still have an active secondary market but they are probably not profitable at market rates for inference because newer chips are so much more efficient.

If Semi-Analyses is correct [1], even H100s would lose money at current prices - I doubt that is actually true of H100s in practice because from what I can tell it is based on a 3-year depreciation from the new price, but the point is that there is a point at which old hardware's operating inefficiencies dominate.

https://inferencex.semianalysis.com/infx/profit-estimator/de...

> since everyone is assuming it's a winner take all market. That's probably close to correct.

why? I keep seeing this over and over again but I've not seen an explanation. I would really appreciate some insight into this line of thinking, since it's not obvious to me. What is the benefit of holding hundreds of billions of computer hardware when it's not proven that it's even needed.

The idea is, innovation will concentrate where the necessary ingredients are.

When AI replaces human workers and researchers, the "frontier", with which you can earn money and gain an edge over your competition, resides in those data centers with the highest concentration of computational power.

If you own those data centers, you control the frontier of human knowledge, the edge where money and power is being carved out.

Lets say the total market for an arbitrary winner-take-all industry is $100 Billion/year in profits. Just so that we have a round number. And for another round number, lets assume the fair market PE ratio is 20x. This over-simplification would then have a fair market valuation of $2 Trillion total.

Just simple napkin math.

Now, lets say there are 5 companies competing to be the winner. Only one can be the winner, but all 5 are trying to be the winner. How much should each company individually spend?

Well... starting-bid is $2 Trillion, or ~$10 Trillion in the aggregate. Its basically the waste of competition. Because everyone is vying for monopoly power, the only fair for them to all invest over $2 Trillion in the hopes of being the dominant one at the end. Even if $8 Trillion is wasted, that's "other companies", your competitors, who lose out.

But wait: if one company spends $2.1 Trillion, that one company likely becomes the winner and the other 4 companies lose out. So then another company spends $2.2 Trillion, to get ahead of that war. Etc. etc.

The gamble: gunning for market supremacy before investor cash dries up.

The real gamble: betting there is a large enough pool of c-suite that doesn't fundamentally grasp the concept of vertical integration that they're willing to pay to train their competition. Because if you think AI companies aren't training models how to do everything your company does with your data you are deeply delusional.

The real real gamble: praying sovereign governments stay anesthitized to the national security implications of a handful of extra-national organizations operating the conputer security equivalent of WMD with no oversight.

Fun part ... all those GPUs are going to shredder after use.

but is it a winner takes all market? currently we have models that are close-but-not-quite SOTA which are still heavily used and very profitable because they are _good enough_ (i'm talking about deepseek, of course).

or is the "winner takes all" mentality based on the assumption of being the first to achieve the technological singularity?

> or is the "winner takes all" mentality based on the assumption of being the first to achieve the technological singularity?

Alternatively, it may be based on becoming the favored provider of the defense or intelligence industries.

Everything happening right now hinges on the idea that there's a singularity approaching.

If you think that, then all this greed and stupidity makes sense.

If not, well, it's just greed and stupidity init.

You seem to be wrong about this

> A typical one-gigawatt AI data center requires $38 billion in up-front capital expenditure (CapEx) and $0.9 billion in annual operating expenses (OpEx). If CapEx is annualized over each asset’s lifespan, the total cost of ownership equates to $8.5 billion per year. Servers dominate this cost at $5 billion per year, or 60% of the total. Operating costs are small by comparison: even energy, the largest OpEx category, costs only $0.6 billion per year.

https://epoch.ai/data-insights/ai-datacenter-cost-breakdown

1 Gigawatt * 1 year = 8760 GWh * 0.8 load factor * $0.2 per kWh = $1.35B of only electricity.

Yes, and that is small compared to the building cost.

They're paying way under 20 cents a kWh. They're paying less per kWh than residential households. If the residential rate in an area was 20 cents, I'd guess the datacenter is paying 40-60% of that rate

They get bulk discounts because their load is 24x7, so the fixed costs covered by a per kWh charge is spread over many more kWh than a residential customer

This is my heuristic for the capex split and effective lifetime of the asset it creates (I'm no expert of course, this is based on various reading):

- 20% Capex has ~20-year lifetime: The shell (land, building, cooling gear, rack mounts, power lines etc.).

- 80% Capex has ~3-year lifetime: The computers, including bulk commitments to power and maintain them.

And interest rates are why this split is a ticking time bomb. The sheer quantum of investment to build one is almost entirely debt funded (~90% of total outlay). I would guess the endgame would be bad even if the split for these entirely different asset classes is 50/50, which would be very generous, based on what I've understood.

Datacenter funding is extremely leveraged. For comparison, only some banks are allowed that much leverage for that size of capital, that too only if they have federal reserve backing. Even in such a case, the retail account holder (pseudo-investor, because our deposits are loans to someone else) has practically no downside protection. If the bank collapses you will lose everything.

Thanks for pointing that out. That has been my concern as well. Sure the post says "they make $2.08 for every $1 of compute" but that seems to only be accounting for OPEX. If we start accounting for CAPEX and variations with interest rates things look a lot less rosy. I mean a lot of folks are suspecting Anthropic will get a junk bond rating when it rolls out to public markets.

I definitely get "Community-Adjusted EBIDTA" vibes from that number.

(for those who don't remember WeWork, this was their made-up variant of EBIDTA that they tried to use in their S1 filing before crashing out, here's a random article on it https://privateequitymodels.com/blog/community-adjusted-ebit...)

This appears to be from this weird graph on page 137 of their S-1, "Significant increase in revenue per dollar of compute". It's not really explained well and has very not only WeWork but also Enron and even DotCom ("number of eyeballs on the page") vibes.

It _could_ be the cost of running inference to respond to requests which bring in revenue, however the cost is described as "compute and infrastructure expenses", while revenue is described as "in-period revenue". I don't see these numbers matching anything else in the filing, however I do not claim to go over the entire document.

So to be perfectly honest I'd just ignore this metric completely :)

Even if we allow for the strange “it’s only a one-time cost” hypothesis, the author’s next paragraph outlines another actual bubble, just 5-10 years from now:

> Once all the data centres are built, AI spending is going to plummet.

Yeah and what happens if none of those data centers translate into more clients? Its not a one off, its a gamble too.

Please, Giancarlos. Let's all behave with the times... I think it's culture-appropriate to say "You're absolutely right. Datacenters aren't more clients. We build. What if they don't come? It's not just a one-off. It's a massive gamble."

(edit: fix name)

To be clear: it's a tongue-in-cheek riff, macroexpanding Giancarlos's (I think humorous) comment. I don't mind the downvotes---acerbic / black comedy is risky business :sweat-smile:

To be fair, I could be wrong, we could all be wrong, we don't know what we don't know. On the other hand, I do think they should come up with a better plan than just trying to retrofit them everywhere they can find. I do think some states make more sense than others.

We will know once it all unfolds, on the other hand, at least there's people who are getting paid to build these.

Worse case, we have acccess to (hopefully) really well-built data centers that can be used for anything hosting related in the future.

I'm hoping the "worst case" you state becomes real, because that would be a great case! Blogged it here:

After "AI": Anticipating a post-LLM science & technology revolution

https://www.evalapply.org/posts/after-ai/index.html

I, for one, welcome the coming age of the post-LLM-datacenter-overinvestment-bust-fueled backyard GPU supercomputer revolution.

(edit: in which case, I'd say more people would get paid to build more of these, except not as obnoxious township-destroying behemoths ... the community library, as opposed to the giant book seller in the cloud)

(edit 2: And I'd go as far as to say that it is anti-employment-generation to allocate economy-scale capital to a very small number of very large scale industrial installations... You wanna incentivize a large number of small-to-medium ones instead, that employ locally in many many more places. That is how you win back industrial capacity.)

The main problem is that once training is complete, you only need inference which needs far less compute.

this also misunderstands the data center rollout itself. openai and anthropic etc aren't building any data centers. they're promising to buy billions of compute and 3rd party companies are taking those promises and using them to fund the construction of data centers which will then lease out that compute back to them. so it's not actually a capital construction project at all. CoreWeave or whoever is making the capital investment. OpenAI is just promising to make that a good investment.

I mean that's not entirely true

https://openai.com/index/building-ai-infrastructure-with-the...

Yeah, Open AI and Anthropic may be buying the bulk of their compute from other providers but they are now moving into building their own infrastructure directly. There is a scale beyond which it makes more sense to build vs. rent

GPU seem to last lot longer than expected.. And have good resale value for other use cases. The $2 for every $1 inference cost quote from Anthropic I presume includes all opex.

That's because there's a temporary supply side crunch.

That only explains why the price of those GPUs at resale is high, but it doesn't invalidate the fact that those GPUs are still very useful way past a 2 to 3 years depreciation schedule advocated by Michael Burry et al.

Older GPUs can still be used to run model inference, albeit more slowly than newer GPUs. We already see frontier labs turning this insight into an opportunity to charge a premium on token delivery speed, when models run on newer hardware. As AI becomes more widely spread throughout the economy, not all workloads will require fast inference, and there will be many use cases for previous generation hardware. The only exception is if the latest batch of specialised chips become ubiquitous very quickly, but then price of compute AND intelligence per watt will both go down dramatically, which would reduce the need for sustained high capex and opex

Yes it is a supply-side crunch (chip manufacturing), but IMHO, it is not temporary. Only two things will remove the supply constraint.

- Demand drops like a stone: raw TFLOPS / per model. This will happen if someone figures out insanely fast and power-efficient math (specifically, multiplication) for existing chip architectures, that makes it dirt-cheap to keep re-computing ever-better SOTA LLMs, as well as to on-line compute inference with ever-bigger models (scaling laws rule). These are different workloads, AFAIK.

- Moore's law re-starts, because someone figures out entirely new transistor designs that can deliver at least up to 100x per-core frequency gains over Si transistors (which would be nothing considering that Si transistor tech delivered ~1Mx gains since invention) and which can be manufactured at similar bin sizes, at similar scale (i.e. equivalent per-die yield), price-competitively (no more 10x the equivalent in SOTA Silicon compute).

I'd say that a math breakthrough might be more accessible than the transistor-manufacturing breakthrough.

> Moore's law re-starts, because someone figures out entirely new transistor designs that can deliver at least up to 100x per-core frequency gains over Si transistors (which would be nothing considering that Si transistor tech delivered ~1Mx gains since invention)

That assumes that there is no hard, physical limit on Moore's Law, which is obviously false. There is a limit, and as long as we depend on mass flow of charge through conductors, it's nowhere near quantum sizes.

Where that limit is has not yet been determined, but at this point the support structure to write transistors is the size of a house trailer, using a staggering amount of toxic chemicals (just to run the exotic lasers). So, advancements today may scale very nonlinearly in cost.

The very phrase "Moore's law restarts" acknowledges that we are no longer on-track.

Yep, I mean to imply practically impossible, in context of bubble-timelines. Theoretically anything is possible given enough decades. Practically, AGI has thus far been 20 years out, as has "clean" fusion power, 1 gee EM drives etc.

IDK if Moore's law restarting is possible. But I wouldn't rule out some sort of non-silicon compute breakthrough that functionally jumps us a couple orders of magnitude. The entire world economy being aggressively focused on compute seems like a good environment to drive investment into figuring out photonic computers or whatever. We do know at least one other non silicon highly energy efficient compute technology (biological brain) exists and works very well

I reckon it depends on how hot they run GPUs for how long. Running any chip at clock or overclocked (likely) will burn it out faster than "business as usual" workloads.

Defraying of capex also depends on how the hyperscalers scrap used hardware.

AFAIK, all hyperscaler hardware is effectively a consumable. Disks, memory, boards, chips; all the high-value pieces do not get refurbished into new computers for sale into secondary markets. The commercial and legal liability is too high---leaking their own proprietary IP as well as leaking customer IP, not to mention secrets and hardening configs and so forth.

I can't recall where I read it, but if a datacenter is modeled as a bunch of black-box hot-swappable shipping containers, then for pre-LLM workloads, per-container unit economics works only as long as a container delivers over 30% capacity. After that you got to swap it out, and send the black box into a furnace to recover them rare earth metals for the next generation of chips.

To a first approximation, high-throughput, gpu-maxxing math workloads (LLM inference) are surely harder on chips and memory, than are bread-and-butter SaaS / data lake / ETL / RDBMS workloads. Speculating further, I'd guess that the ~40% operating margin of datacenters is due to the relative compute-friendliness of non-math workloads for lack of better words.

> Running any chip at clock or overclocked (likely) will burn it out faster than "business as usual" workloads.

Could it be that they _have_ to run them like that because of the shortage of compute volume and once there are sufficient data centers, they can be run relatively cooler and thereby extending their lifespans? That doesn't bode well for NVIDIA/AMD.

> AFAIK, all hyperscaler hardware is effectively a consumable. Disks, memory, boards, chips; all the high-value pieces do not get refurbished into new computers for sale into secondary markets. The commercial and legal liability is too high---leaking their own proprietary IP as well as leaking customer IP, not to mention secrets and hardening configs and so forth.

Maybe that could be a catalyst for going for more "commodity" approach to hardware instead of highly spcialized (atleast for some sections of the hardware) so that the IP is not the hardware but the model/software and they can re-coup the costs by selling the hardware to "tier-2" providers that might emerge based on using cheaper second-hand hardware?

I suspect GPU and CPU failure rates have more to do with stress from thermal cycling, if they can keep them loaded and running without heating/cooling cycles, they should last pretty long. Their lifetime is probably dictated more from when do they stop being competitive on a flop/watt basis than by their failure rate.

Interesting... Upon reflection, I feel like it's a "yes, and" proposition coming down to failure mode.

[Type A] Constant use a constant power level: Quantum world rules would dominate.

When run hot, electron tunneling will kill off individual transistors, and AFAIK that is a cascading failure mode once it takes root. A big enough failed spot, even if localised, could effectively kill a core (if not the whole chip---too many errors). A local hotspot will directly affect neighbours, and indirectly the chip as a whole because the same work is being pushed on fewer cores.

[Type B] Thermal cycled: Mechanical and electrical factors would dominate...

Microfractures / mechanical damage in weak areas of whole chips, and motherboards. In a hyperscaler server rack, if a board dies that part of the rack is dead---micro-maintenance is probably too expensive.

And speculating further, it stands to reason that Type B failures would likely cause Type A failures, and vice-versa.

> Running any chip at clock or overclocked (likely) will burn it out faster

server parts are tuned for power efficiency and longevity

B200 has an operating range of 10-35C. https://resources.nvidia.com/en-us-dgx-systems/dgx-b200-data...

RTX 5090 (and most consumer CPU/GPUs) have thermal limits at 90-100C.

> The $2 for every $1 inference cost quote from Anthropic I presume includes all opex

No, it does not, its just revenue minus compute. Per their S1, Anthropic has never turned an operational profit.

That's simply not true. See page 160 of the S-1 - clearly shows positive operating profit in Q2 2026.

> And have good resale value for other use cases.

Do they? Is H200 $-competitive with an ASIC for crypto? Bearing in mind they're 4 figures.

Zero cash flow is not the same as zero earnings. They may end up with negative earnings sometime in the future depending on how much additional capex they spend, how long they take to depreciate the capex, how fast revenue grows, etc

because of GPU depreciation schedules and competitive hardware upgrade cycles), and opex of delivering LLM inference services, and being able to price it all competitively.

Hacker News increasingly feels like being a horse in 1916, reading in the Horse Bulletin that Ford is a scam and a bubble because the mass-market automobile will soon be commoditized.

There's an object-level claim to consider, but behind it the more baffling implication that this would be good for horses.

No horsing around here, just money math that says interest rates on debt sunk into datacenters will far outstrip the most optimistic revenue projections as of 2028-29-ish, when the bulk of the debt will come a-calling.

Also, inverse Pascal's Wager, because horse v/s motorcar is a false dichotomy... they are bound by the same laws of physics (faster horse, less shit, more noise and more smoke), merely different economic realities.

Hyperscaler bet is "Superintelligence" and/or "AGI", or we will make a God. That is a technological singularity. There will be a "before God" and "after God" world, and we will very likely never be able to understand what the AGI/Superintelligence's world looks like. (Because to understand what it is like to be a bat, you have to become the bat, but at that point your meaning of "understanding" itself changes entirely).

So I believe ten trillion dollars and a bunch of pirated books and source repos, and well pirated everything, mostly of very dubious quality and provenance, will not result in a God. I've been acting accordingly.

I mean Ford didn't get a trillion dollars because he is to be only manufacturer of any and all transportation. There are still horses, and Ford and Dodge(Anthropic) are not even the biggest manufacturers.

Reply on news.ycombinator.com