> Of course an LLM does have a latent concept of its own confidence that is internally consistent, and is perfectly able to articulate it
Can you provide credible reference for this?
Can you provide credible reference for this?
It is given sets of paramaters/data and infers a response based on it's acceptable confidence level. The bottleneck for this is compute power and model/prompt/rails/integrations.
The reason LLM stays consistent/can be somewhat trusted to stay on rails is that it's aware of both it's context/scope and it's confidence about the context.
Anything else would be....original thought and exposition? I guess. You could say that our brains do the same, IE; unconcious statistical analysis leading to inference based on confidence.
Similarly, a language model trained to emulate speech, follow instructions, and solve problems will not implicitly have an understanding of its own statistics. The research doesn't back that up and it just doesn't make any sense if you think about it.
The only thing LLM produce that could be considered its "confidence" would be its token logit values, but even that is a sketchy connection.
Any inference or probability engine, be it an app or a heuristic mental process, is obviously based on categorizing historical data and literally weighing it's confidence around it to form responses.
Confidence intervals and associated indexes/logic are the foundation for most heuristic data models.
Does the output of the model have insight into those details? Just because our brains function based upon electrical potential doesn't mean we have innate insight into it.
Certain models of course force this context memory, some don't, some allow you to toggle it's "trigger level".
For example Anthropic, use haiku for low memory context queries, research. Then Opus/Fable for compiling/analysis/design, which relies on the LLM inference constantly going back into context memory.
Another example is OpenAI/GPT. The "free" tier forces context memory using the "highest available model" per your subscription (or lack thereof), and upon hitting its limit, drops you down to a low context model, and then ultimatley blocking that chat entirely.
(I appreciate this discussion and am not trying to inform you, I'm sure you already know, but typing these things out helps me understand my holes in understanding.)
> It is given sets of paramaters/data and infers a response based on it's acceptable confidence level. The bottleneck for this is compute power and model/prompt/rails/integrations.
or whatever the LLM claims its confidence is on any given output?
A good litmus test for this is, typically any artificial neural network, perceptron or even a simple linear regression model do statistical inference based on weights and selection based on confidence level. It's a generic characteristic, which doesn't explain anything specific to LLMs.
> The reason LLM stays consistent/can be somewhat trusted to stay on rails is that it's aware of both it's context/scope and it's confidence about the context.
Post hoc ergo propter hoc.
> I guess. You could say that our brains do the same
I understand that it "feels" the same from a certain perspective, but we don't know that.
"It's aware" is a big leap and bold original thought, as awareness is not something it has full stop, try different terminology?