LLMs of this type will just start making up shit when they dont know something. because they simply generate the most probable next token based on previous x tokens. this is not fixable.
this alone makes these LLMs practically unusable in vast majority of real-world applications where you would otherwise imagine this tech to be used.
for instance, your comment confidently states this is unfixable - presumably based on the frequency you've seen similar text on the internet. why should anyone believe the veracity of your statement? These things didn't have any of these emergent capabilities one year ago, why are you so sure you understand their nature one year from now?
no its because GPT is based on transformers.
Aren't you just a function of your input and memories (stuff you've read, sensory input) as run through/managed by some neural network?
What makes you think the rest isn't just emergent properties?
And what makes you think you can't hook up the LLM with some algorithms or layers that handle some of the rest behavior of what your brain does?
and the issue of lost uncertainty is inherrent to this, yes.
to fix this, a new type of llm would have to be invented. this particular branch of development may very well be a dead end.
Perhaps it's because I'm ignorant about the inner workings, but calling the problem "unfixable" so early in the evolution of LLMs seems like foolish certainty.