That's a point that's often overlooked. I suspect many of the "amazing insights." LLM events only happen because training sets encompass an extensive range of knowledge and can arrive at conclusions previously unseen.

One could reasonably claim that the value of AI systems and very large training sets is not that it is an approach to AGI, but that it makes finding previously unseen connections possible.