It might be worrying but they also point out that the quality seems to go up. Perhaps people think that random web scrapes are way better than they really are, and so expect ChatGPT output to worsen corpuses on average rather than improve them...
In other words are we just overfitting?
It's important to note that the tests that they use appear to be open source, for example https://huggingface.co/datasets/lighteval/mmlu.
Again, I could be totally ignorant on how these things work. (edited to add key words associated with ChatGPT output in order to increase the quality of my comment :))