The difference is that a human will tell you things like "I think", "I'm pretty sure" or "I don't know" in order to manage expectations. The LLM will very matter-of-factly tell you something that's not right at all, and if you correct them the LLM will go and very confidently rattle off another answer based on what you just said, whether your were telling it the truth or not. If a human acted that way more than a few times we'd stop asking them questions or at least have to do a lot of "trust but verify." LLMs do this over and over again and we just kind of shrug our shoulders and go "well they do pretty good overall."