Does the output of the model have insight into those details? Just because our brains function based upon electrical potential doesn't mean we have innate insight into it.
Certain models of course force this context memory, some don't, some allow you to toggle it's "trigger level".
For example Anthropic, use haiku for low memory context queries, research. Then Opus/Fable for compiling/analysis/design, which relies on the LLM inference constantly going back into context memory.
Another example is OpenAI/GPT. The "free" tier forces context memory using the "highest available model" per your subscription (or lack thereof), and upon hitting its limit, drops you down to a low context model, and then ultimatley blocking that chat entirely.
(I appreciate this discussion and am not trying to inform you, I'm sure you already know, but typing these things out helps me understand my holes in understanding.)
> It is given sets of paramaters/data and infers a response based on it's acceptable confidence level. The bottleneck for this is compute power and model/prompt/rails/integrations.
or whatever the LLM claims its confidence is on any given output?