Is that really human input, or just grounded input in general?

Most human-written texts on the internet refer in some way to the real world, or at least to things outside the text itself: They might be descriptions or reactions to current events taking place, or blog posts or wiki entries discussing some aspects in detail. Even completely fictional texts reproduce basic assumptions about the world, culture markers, etc.

In contrast, LLMs learning on their own output have no connection at all to the outside world, the data just reinforces all assumptions the model already has, whether or not they are true or useful.