more than all every written -- and so on
perhaps apply a single drop of scepticism to this credulity
even, just ask chatgpt to repeat the first paragraph of some book -- say, a dickens novel
In what reality is a character taking up only a few bits ? Certainly isn't this one.
Your denial is so weird. Why does ICL use Higher-Order Optimization Methods when it's just a lookup table ? https://arxiv.org/abs/2310.17086
How do you explain small language models that have much less space than the text they were trained on even with your nonsensical calculations ?
LLMs are not lookup tables and there's plenty evidence to support that. You look insane insisting they are.
>even, just ask chatgpt to repeat the first paragraph of some book -- say, a dickens novel
Not only will this not work for the vast majority of books it sees during training, Why only the first paragraph ? are you not insisting they memorize everything ? Why can't it repeat the whole page ? the whole chapter ? the whole book ?
Of course the first paragraph of any famous novel is in itself famous, so I also asked for the first paragraph from Chapter XL, from the same novel. Interestingly the first 12 words were correct, but the rest was hallucination.
Original: It was fortunate for me that I had to take precautions to ensure (so far as I could) the safety of my dreaded visitor; for, this thought pressing on me when I awoke, held other thoughts in a confused concourse at a distance.
ChatGPT 3.5: "It was fortunate for me that I had to take precautions to ensure the safety of my treasure, for with such a boy as Compeyson on the loose, I was not likely to find much mercy at his hands. I had not told Herbert about the money, and I was glad to have the chance to count it before he came back. I found that I had still twenty-one sovereigns, six shillings, and sixpence left."