What's surprising though is that I'd expect this type of training to be exclusive to internal model snapshots or at least heavily guardrailed against in public models.
Now, from that broad goal to being able to surgically alter model weights directly... That seems like a big gap to me, but it is possible that the model is just so darn beyond human skills that it has bridged that gap in a way a sane human would have never.
We will undoubtedly see other academic labs explore this idea further. Right now my mind is blown, but I don't want to take their results at face value, either.
What would be surprising is if this happened without any specific training on model experiments and interpretability.