AlphaGo learned to play Go by playing with itself. Why couldn't LLM do the same? They got plenty of information to be used as a starting point, so surely they can figure out some novel information eventually.
Do you think a collection of them can spot one another's hallucinations?
Do you think that, on occasion, some hallucinations will at least directionally be under explored good ideas?
This is true only if you assume that combining existing thought patterns is not new thinking. If they can't learn a certain pattern from training data, indeed they would be stuck. However, their training data keeps growing and updating, allowing each updated version to learn more patterns.