Comment by suprjami | Hacker News Reader

Comment by suprjami | Hacker News Reader

Parent Full thread

It's stated in many places that inference is profitable (margin 70% to 90%), only research is expensive. Hyperscalers are burning through their cashflow training new models while inference-only providers are printing money.

If all the research went away tomorrow, people are still going to sell inference at current prices or higher. There are already open weights competitive with proprietary models. There will be no lost capability.

If businesses find inference useful today then it doesn't have to drastically improve in a short timeframe anymore. History is full of inventions which became "good enough" and didn't improve much or at all for a long time.

eg: Western society runs on radial tyres which have seen only marginal improvements for the last 50 years.

(yes there have been some small improvements in compounds, tread patterns, TPMS, etc. hardly drastic revolutionary changes to the tyre industry)

> It's stated in many places that inference is profitable (margin 70% to 90%), only research is expensive.

It's been stated by the AI companies, which are known to routinely lie to our faces, and have a vested financial interest in making people believe they will be able to turn a profit at current prices. In other words, it's one of the most unbelievable claims out there, and you shouldn't believe it for a second.

Even assuming that inference is really profitable today's models don't learn new things by themselves, except in the limited sense of temporarily storing everything they need for a conversation in their context (and maybe leaving themselves little notes in md files like the guy from "Memento").

The difference with tyres is that If today's LLMs had been invented and had become "good enough" 50 years ago, you would have a cutover in their knowledge that excludes 50 years of information. Every "write me a program in Rust" conversation would involve LLMs filling up their context trying to learn Rust programming from scratch every time and probably doing a very bad job.

An example of that was when Fable disproved that mathematical conjecture and HN was full of other people who fed that information to other models (or Fable itself) and received incredulous answers from their LLMs. If something is proven true or false in math, the world of science moves on and that new piece of information can be used to build, prove or disprove other things. But an LLM is excluded from learning even from the very thing it just helped demonstrate and needs to re-discover it over and over again. In order to have an LLM that "lives" in a world where the Jacobian conjecture is false, you need to train a new model and add that information.

Reply on news.ycombinator.com