LLM scaling laws are pretty well established at this point. They probably won’t hold forever but we aren’t at the breaking point yet.

Some more pressing questions are:

* What new capabilities emerge as the models get better and better at predicting (i.e. loss goes down)?

* How much will it cost to train increasingly large models? And to run inference on them?

* How difficult will it be to find or generate more and more high quality data?