Sadly it's not even just references, LLMs still hallucinate or at least misrepresent even the most basic of facts. That and the stereotypical GPT-verbage makes it impossible to use for writing anything significant.
Keep in mind that there's no difference between what happens inside a model when it "hallucinates" vs. when it generates "correct" output. It's the exact same process.
Surely when google gives bad results, it's "the same process" as when it gives good results. And when a book gives wrong information, it's the exact same kind of ink as correct information.
I would imagine there's also a raft of people who will use it as a reason to give up on any search for truth.
I still do hold a lot of hope for their eventual capabilities, but I'm also pretty pessimistic on what the direct and Nth order social effects will be.