Hacker News Reader

however you just have to backtrack every once in a while because the information turned out to be false.

The problem is, how do you know? I've seen developers go completely off-course just from bad search engine results and one did admit he felt something wasn't right but kept going because he didn't know better; now imagine he's being told by a very confident but incorrect LLM, and you can see how hazardous that'll be.

"You don't know what you don't know."

Unless you have a very good understanding of the system you’re working on or the tools you’re using, it’s very possible to get knee deep in crap without knowing it. That’s one of the biggest risks of using LLMs as assistants.

You need to develop skills like critical thinking, metacognition, analytical reasoning - being able to get to a robust mental model from a bunch of different inputs, some of which may even contradict each other.

People were generally already horrible at that before AI.

And therein lies the problem - if you're not already an expert there's no way to tell when is the right moment to backtrack.

The exact definition of a useful heuristic, "good enough"