I wonder how far this can be taken. Could Qwen 3.8 can interpret and make improvements to Qwen 3.5 0.8B weights, or even its own weights?
You can optimize a model by randomly changing its weights and validating results on your task. A monkey can type a book given enough attempts.
For training you would need to hook it up to a LoRA training harness, identify the looping weight direction, and adjust the weights accordingly. I've not (yet) looked into how to do this.