Hacker News Reader

The math example on Qwen was based on 50 batched iterations, and the fix was essentially to remove a direction in activation space that causes looping instead of an exit, this was already known to be a thing.

So it's not able to insert new knowledge, but it's able to adjust the routing/looping/weighing of existing one.

As I understand, it can replace post-training like LoRA/GRPO, but if it'd cheaper the authors don't go into

View on HN

I don't think its that crazy that you could look at activations and tweak weights directly to get a certain outcome, but isn't the question whether you can do that without side effects or degrading/breaking other parts of the model?

If I want to do that in a generalizable way that preserves performance across all domains then am I not just back to training again?

Yep youre right, I don't see an eval on this.

I think you could still essentially have a really cheap Lora-Post training on a frozen base model by doing this. And instead of creating 200 checkpoints for different post-trained directions just adjust a single one on the fly.

Reply on news.ycombinator.com