Hacker News Reader

Comment by dweinus | Hacker News Reader

Sure, but you'll have to pay for what it really costs

I thought the incremental supply cost per token wasn’t that high, and the main expense for these companies was the training they do on top of that?

If the capital cost gets written off, because it can never be recouped, then there’s only the operating cost left.

Suppose OpenAI goes bust tomorrow and all those data centres get sold for pennies on the dollar. Whoever buys them can operate them at cost.

Inference will keep getting cheaper and cheaper though. Perhaps model releases will be fewer and farther between because these groups can't affording paying out a billion or more in training each generation

The chinese models are being trained for less than a million a pop.

Yeah, but Wright's law (cost of making the n^th something (like a token) being exponentially less than the cost of making the first) strongly suggests that post-bubble inference will be significantly cheaper the longer the bubble lasts.

Whether that's a good thing considering the other things those dollars could be going towards...

I mean I'm happy for better but it might start grounding the industry where 1.6% better on some benchmarks doesn't justify hundred million investments.

Did internet get more expensive after the dot com bursted?

Yes! At first there free, yes free, ISPs. Then remember when news sites didn't have pay walls? When every site wasnt loaded with ads and trackers? When Facebook, Twitter, or linkedin provided open data apis or rss? The enshittification came for it all because the bills were due.

.. which will be the same you are paying today. Token prices are already above their operating costs.

If you're talking about free tiers, yes, that might change as marketing budgets will shrivel up.

I have only ever paid per token for this reason: if I can’t generate sufficient value to cover the real cost I don’t want to invest in using it

Reply on news.ycombinator.com