Hacker News Reader

Unless I misread it, this article simply doesn’t mention training or the cost of it.

Yes, and the training data behind a lot of models is already stale and rotting fast.

Maintenance is just as important and the cost is often overlooked. The sober reality is that an LLM is just a natural language search index. For now, Google might be the only one with the capability to maintain "training" at the pace most people expect.

Training would fall under "compute and infrastructure investments", which I mention in the 3rd paragraph.

You mention that training can be suspended, which is true, but don't acknowledge that puts them behind in the arms race against both other frontier companies as well as second line cheaper providers.

Without a moat, training is an ongoing cost that they can't afford to not pay

Imagine if Apple decided to "stop developing new iPhones". That's what "stop training" sounds like.

They'll stop training when they're dead, not before.

Training is not infrastructure. It’s not capex, not “investment”. It’s really an ordinary running cost of what they do, for the foreseeable future, and it’s absolutely enormous. They couldn't even dial back training in a winner-takes-it-all scenario, because training of smaller, domain specific models would remain a way to compete.

Reply on news.ycombinator.com