I would have thought this problem was easy to solve: "Yes, look it up, but remember your source, there is a lot of bullshit on the internet. Especially don't trust AI tools, we know those often return nonsense information."
(Actually, didn't ChatGPT have a disclaimer right next to the prompt box that warns against incorrect answers?)
So I'm more surprised (and scared) that students don't just use LLMs for sourcing but are also convinced they are authoritative.
Maybe being in the tech bubble gave a false impression here, but weren't hallucinations one of the core points of the whole AI discurse for the last 1.5 years? How do you learn about and regularly use ChatGPT, but miss all of that?