Comment by fpaf | Hacker News Reader

Comment by fpaf | Hacker News Reader

Parent Full thread

If you are using AI like you use Netflix (something you like but can live without if it gets too expensive) then you have a point. But if you are relying on AI for your business, the cost of AI sooner or later is going to be passed on to you and it's going to impact your margins. And it's probably goong to be sooner, rather than later.

The amount of money these AI companies are burning is unprecedented and there is simply not enough money in the economy to keep it going at a loss for ten years. Google (Google!) went cash flow negative and is issuing bonds, basically asking for a loan. That's the reason markets are getting so nervous.

And yes, choice is good but how many of these new models are trained from scratch vs distilled from frontier models? If OpenAI and Anthropic go down in flames how much of the cheaper choices we see today are going to keep evolving? I am not trying to make a prediction either way, I am just pointing out the uncertainty. Which, again, is fine if AI is a commodity you can easily cut, but not great if your company is betting big on AI.

It's stated in many places that inference is profitable (margin 70% to 90%), only research is expensive. Hyperscalers are burning through their cashflow training new models while inference-only providers are printing money.

If all the research went away tomorrow, people are still going to sell inference at current prices or higher. There are already open weights competitive with proprietary models. There will be no lost capability.

If businesses find inference useful today then it doesn't have to drastically improve in a short timeframe anymore. History is full of inventions which became "good enough" and didn't improve much or at all for a long time.

eg: Western society runs on radial tyres which have seen only marginal improvements for the last 50 years.

(yes there have been some small improvements in compounds, tread patterns, TPMS, etc. hardly drastic revolutionary changes to the tyre industry)

> It's stated in many places that inference is profitable (margin 70% to 90%), only research is expensive.

It's been stated by the AI companies, which are known to routinely lie to our faces, and have a vested financial interest in making people believe they will be able to turn a profit at current prices. In other words, it's one of the most unbelievable claims out there, and you shouldn't believe it for a second.

Even assuming that inference is really profitable today's models don't learn new things by themselves, except in the limited sense of temporarily storing everything they need for a conversation in their context (and maybe leaving themselves little notes in md files like the guy from "Memento").

The difference with tyres is that If today's LLMs had been invented and had become "good enough" 50 years ago, you would have a cutover in their knowledge that excludes 50 years of information. Every "write me a program in Rust" conversation would involve LLMs filling up their context trying to learn Rust programming from scratch every time and probably doing a very bad job.

An example of that was when Fable disproved that mathematical conjecture and HN was full of other people who fed that information to other models (or Fable itself) and received incredulous answers from their LLMs. If something is proven true or false in math, the world of science moves on and that new piece of information can be used to build, prove or disprove other things. But an LLM is excluded from learning even from the very thing it just helped demonstrate and needs to re-discover it over and over again. In order to have an LLM that "lives" in a world where the Jacobian conjecture is false, you need to train a new model and add that information.

Reply on news.ycombinator.com