The math example on Qwen was based on 50 batched iterations, so the model was able to evaluate it's changes on downstream tasks. And the fix was essentially to remove a direction in activation space that causes looping instead of an exit, this was already known to be a thing(all of the jailbreak/unleashed models do this) following Arditi et al. (Refusal in language models is mediated by a single direction, 2024)

So it's not able to insert new knowledge, but it's able to adjust the routing/looping/weighing of existing one.

As I understand, it can replace post-training like LoRA/GRPO, but if it'd cheaper the authors don't go into