But yes, the major issue is that it simply can not indicate what it can not do and simply makes up abilities.
It is a major limiting factor when we can not know the confidence in correct answers.
But yes, the major issue is that it simply can not indicate what it can not do and simply makes up abilities.
It is a major limiting factor when we can not know the confidence in correct answers.
It feels a bit like someone saying ‘this isn’t a fundamental limitation of cameras; the lens is the problem’… but I’m very much ignorant about these things so I’d be happy to have it explained.
> In this case it would be pretty easy for the LLM to reason about how often t appeared in the sentence.
I’m not sure that follows though; is it true that LLMs already find it easy to ‘reason’ about how often (token) appears in some given text? I don’t think so.
Also, tokens are just numbers internally after tokenization. So LLM sees the question as "how many letters N are there in 563 3684 11267?". It has basically no idea what you're talking about. Unless somewhere in the training data someone mentions that token number 11267 has two letters N.
It’s not just the latter part of that sentence that consists of ‘numbers’, all of it does.