I’m not super impressed by this, LLMs are great at brute forcing and analyzing large amounts of data. It’s literally writing a program except in a programming language we don’t understand.
A single forward pass here Bette 10mm+ PPO rounds. You are hard to impress.
Seems like a pretty big step towards understanding the language no?
So they are reading and analyzing the weights and probabaly run some experiments to see how modifying them will make some answers better. makes sense they can do that.