Because we want to use this as a replacement for humans, and the average human can learn the rules of chess without needing to see the rules explained hundreds of thousands of times in millions of games.
So, yeah, it matters if a model has millions of examples of something in its training set and still cannot follow the rules.
Okay, lets go with that: it's the "shown the rules" bit that we are arguing about.
The argument is that a human may play maybe a dozen games after learning the rules, after which they won't be inadvertently attempting illegal moves. What we are observing with SOTA models is that, even after seeing millions of chess rules, rulebooks, actual games, etc, they still attempt illegal moves.
This does not point to generalisable and adaptable intelligence, such as we see in the average human.
See my comment here for more: https://news.ycombinator.com/item?id=49725306