AI Scaling Laws

SemiAnalysis consistently does deep technical posts like this. Worth subscribing.

My notes:

Scaling is continuing. Amazon's 400k trainium2 chips, Meta's 2gw datacenter, OpenAI's multi-datacenter training.

Opus 3.5 training succeeded. But it's a more profitable decision to use it to train Sonnet 3.5 and serve that instead. Large models are now teachers, not necessarily end products. Too expensive to serve to end users vs what they'll pay, but great for improving smaller models that are cheaper and faster to serve.

Orion (GPT-5) is being used for training data generation and in verifier/reward models. They say it's not economical to serve to end users until Blackwell chips (B200).

Models that can explore reasoning chains get smarter on certain kinds of problems. [My note, not from article: Math, science, law, programming. R&D, law and programming are perhaps the industries that are willing to pay more for higher reliability.]

Scaling with "berry training" - monte carlo tree search generating thousands of different answer trajectories, then uses functional verifiers to get rid of the ones that didn't arrive at the correct answer.

Big focus is on making inference cheaper and faster. [My note: If you want to work in AI, I imagine any research on LLM inference cost and speed will be highly valuable.]