Comment by I_Am_Nous | Hacker News Reader

Comment by I_Am_Nous | Hacker News Reader

Parent Full thread

For me, it adds a shadow of doubt on the reliability of a peer reviewed journal if one of the "peers" is an LLM during the "it doesn't even know it's lying" AI stage we are currently in.

I read a review of The Singularity is Near by Ray Kurzweil where it was described as seeing a table full of what appears to be very delicious food, but it is then revealed that there is absolutely some amount of dog feces mixed in with some of the dishes. You can't tell which is safe, and which is carefully crafted with dog feces.

An LLM in a peer reviewed journal currently has no place, unless it is part of an experiment where it is trained on the Journal's body of work and then tested for accuracy with future articles. As the tech progresses it may find a place but if it takes twice as long to fact check the LLM output it's saving nobody time and possibly hallucinating in hard to catch ways.

It's even worse that LLMs don't have a concept of lying: they are asked to generate text and they generate it, even if it means making things up or 'lying' as we generally call it. The real issue is the human people treating them as oracular fonts already.

They're as trustworthy as any other oracle then

Or as trustworthy as any other human. Humans often believe things that are objectively not true and yet they will use these things to judge other things by.

That's not a good analogy. Trusting an LLM right now is more akin to trusting a compulsive liar. (Another approximation would be assuming historical fiction was true.)

Humans usually try not to lie, and there's a particular shape to the sorts of details they tend to forget / confuse. Compulsive liars often don't even notice themselves lying. That's closer to an LLM. I trust the output of an LLM about as much as stuff George Santos says.

We seek for our gods in lowly places it seems.

Agreed. LLMs are one of the places where "the math checks out" but that doesn't mean it describes anything helpful, useable, or even correct at minimum.

https://www.cnn.com/2019/10/04/health/insect-rodent-filth-in...

I get what you're saying, but this is exactly the point of peer review though. It wouldn't be worse if the original author was doing shoddy work in some parts.

Reply on news.ycombinator.com