Also agreed. Begging for crumbs at the CoS table is impeding cracking the tough nut of MechInterp. And further facing the fascinating and fertile fact that nobody has a clue of how any of these black boxes really work.-
Just like how we don’t hold entire code bases in our head, but use an IDE to navigate. Or the tools used for searching sequenced DNA.
> There are interesting and surprising behaviors, as they scale up.
... but precisely these "behaviors" being "surprising" (ie. unexpected) are what I am referring to. If we do not understand how these properties emerge, we do not understand how these things work.-
The paradox here as you aptly put it is that the basic mechanisms are relatively simple and understood. This is deceptive. It is in the emergent results where both the promise and the threat are.-