This is pure anthropomorphization. But so it always is with pop sci articles about AI.
This is pure anthropomorphization. But so it always is with pop sci articles about AI.
All I really get out of this experiment is that there are weights in there that encode the fact that it's doing an invalid move. The rules of chess are in there. With that knowledge it's not surprising that the most likely text generated when doing an invalid move is an explanation for the invalid move. It would be more surprising if it completely ignored it.
It's not really cheating, it's weighing the possibility of there being an invalid move at this position, conditioned by the prompt, higher than there being a valid move. There's no planning, it's all statistics.
The chorus line of every human ever attempting to rationalize cheating.
or the whole thing is just a reflection of the rules being incorrectly specified. As others have noted, minor variations in how rules are described can lead to wildly different possible outcomes. We might want to label an LLM's behavior as "circumventing", but that may be because our understanding of what the rules allow and disallow is incorrect (at least compared to the LLM's "understanding").