> about the fact we are dealing with a system that reasons successfully over a raw bulk of extraordinarily opaque data without breaking a sweat. That ability is alien

But it's not what's happening here at all! Read the thing in detail and you'll see that it's not doing anything alien, it does exactly what you (or at least someone with at least some knowledge abouy neural nets) would do if you were asked to perform the task. It doesn't reason about weights in a magical way, instead it reads the easy to interpret parts or use directional ablation to tweak a model's behavior, exactly like how the ERP folks have been decensoring models to produce adult content over the past two years.

It's an interesting demonstration of ML knowledge and agentic ability, but it's not alien NY any means, and it's not actually surprising given how strong today's models are with agentic work.