I don't quite understand this. The returns to scale have always been sublinear (ie "diminishing"), but the scale-maximalists didn't worry about that before. Also I heard Sam tout on the Lex Friedman podcast how amazing it is that the scaling laws worked so well for GPT-4. So I wonder what changed?

Sure yeah the cost numbers are getting very large and we can't keep scaling forever. But Google could easily 10x the training cost of GPT-4 if they thought it would protect their search business. I'm still skeptical that scaling is enough to reach the thresholds we want, but I'm surprised that it's being claimed right now when there's a huge rush of new money into the space. I wonder if this is some sort of misdirection by Sam