Hacker News Reader

Comment by antman | Hacker News Reader

The costs are exorbitant and most software is not produced by companies with such a huge moat. Anthropic made a profit through their recent bait amd switch pricing. There is zero useful insights online to indicate whether this might die due to commoditisation with good enough open models or fail the race to get more people subsidising unsustainable growth with other people’s money. Who knows? In any case they dont seem to be able to drop usage costs so the business model seems based on wishes

View on HN

Continuing with your skepticism:

> Stories are circulating of companies surprised at how expensive their LLM bills are becoming from usage by their staff

> Enterprise customers are now paying API prices

How long before enterprise customers start to question the bill? Anthropic goes from not making money to doing pricing shakeup, and now they are making money and the biggest spenders are shocked at prices.

Seems like things are still very uncertain.

And switching cost is basically zero, so there's not much friction. In my harness it's literally just saying model=X

Usage costs will come down with better hardware. Hardware is improving rapidly each generation.

That trend held true for the past three years, but it doesn't feel as safe to me now.

But memory costs are going way up. And both OpenAI and Anthropic bumped up the price of their frontier models in April.

Algorithms are also improving. I believe it's very unlikely for these two improvements together to not result in one to two orders of magnitude cheaper cost per "intelligence". Of course, that might just make use cases that are too expensive today viable and thereby increase usage further.

Yeah, it’s called supply and demand. Demand for memory went way up suddenly. Now supply is going up rapidly as companies try to cash in on that demand.

Supply will eventually catch up with demand. Then the prices will come back down.

Costs will plummet as better hardware becomes available and priced reasonable so that people can more easily run their own open models locally. But that won't help Antropic/OpenAI make more money, quite the opposite.

A lot of the new hardware requires retrofitting existing datacenters for appropriate cooling, or is waiting to be installed because the new datacenters haven't been built yet. By the time they're installed it's likely a lot of Blackwell GPUs are going to be very out of date. Newer hardware is turning into huge capex bills along with the corresponding depreciation costs. Basically, it's not the same as plugging a new GPU into your desktop, the upfront investment is extremely expensive and all the numbers I'm seeing suggest that the newer GPUs are costing more to run, not less.

Sure, with all the component shortages it’s not surprising the current GPUs are coming at a massive premium.

Eventually either the supply will go up or companies will start buying fewer overpriced GPUs.

Either way, the price per token will come down as hardware improves and supply and demand reach equilibrium.

Technological obsolesence is a bitch eh.

Reply on news.ycombinator.com