> if you give a human the weights of qwen 0.8B and 5 years it'll still be impossible to edit the weights in any legible manner.

Well, is that really so? There is quite a lot of research going on to identify the "meaning" behind certain weights, neurons or directions. I don't see why a human with enough time, perseverance and ability to run experiments with the model shouldn't be able to uncover some of the underlying mechanisms of the model.

But of course not with the speed with which an LLM can do it.