what I am curious about is - what is more expensive, the GPU compute of a traditional fine-tuning loop, or the LLM tokens spent trying to lockpick together a solution. It's impresive - but it definitely feels like it takes a lot of luck and brute forcing with a ton of tokens. Nevertheless, if we are able to train tiny models to be really good at weight-space programming, then the calculus may change.