Ask HN: Why don't frontier AI model providers continuously improve their models?

1 pointjballanc1 comment

Just what the title says: I'm wondering why we're still, years after ChatGPT, having to wait weeks or months for "the next version" of a model when so much else in the software world has moved toward continuous improvement?

View on HN

It just doesn't work that way.

To some extent you can append some knowledge to a model with low-rank adaptation and other techniques but if you want to train a model which is substantially better than your old model you need to train a new model which is much bigger and/or more efficient than your old model and it learns a whole new representation.

See https://en.wikipedia.org/wiki/Catastrophic_interference for one problem.

Reply on news.ycombinator.com