I think that one of the main problems with LLMs is that we're shoveling everything in to them, without any guidance as to what is "real"/"true"/"factual" - crazy anti-vaxxer-cooker stuff is is sitting in their along with science without any concept of scientific reality and no guidance to what is real and what is insane minds spinning on each other