I have yet to find a model that does not make mistakes each turn. I suspect that this kind of error is fundamentally incorrigible.
The most interesting thing about LLMs is that despite the above (and its non-determinism) they're still useful.
I have yet to find a model that does not make mistakes each turn. I suspect that this kind of error is fundamentally incorrigible.
The most interesting thing about LLMs is that despite the above (and its non-determinism) they're still useful.
What kind of mistakes are you talking about here?
We are, in a sense, fallible machines who have designed a planet-wide computational fabric around that fact.
They have cognitive awareness of which tasks are highly critical and need more checking and re-checking without being prompted to think that way.
For a human, time doesn’t stop when the first pass of the prompt and response is over. An LLM effectively wipes its memory of what it just did unless something is keeping track of a highly resource constrained context.
An LLM is like an author of a book that immediately closes its eyes and wipes its memory after writing a chapter. Sure, it can pull some of that back in the next query via context, and it can regain context very quickly, but it effectively has no memory of the exact thing it just did.
When a human is doing these tasks there is a lot of room for mistakes but there’s also a wildly higher capacity for flowing through time.