> well past what we’d expect from humans or conventional algorithms

The model being able to simply type out NN weights, on demand, and have it outperform one trained on 100M inputs pretty much fits that description, doesn’t it?

And being able to prove that knowledge is localized using tools, is also quite different from an LLM simply looking at raw weights and instantly pointing out that 18373 controls left movement.

It’s not a fundamental change in what we know, but a new emergent ability that surely warrants some excitement.