Does it? I’m still unconvinced it’s more than copying other examples of “show your work”.
Does it? I’m still unconvinced it’s more than copying other examples of “show your work”.
And while these LLMs aren't literally just copying verbatim, they are literally just token selection machines with sophisticated statistical weighting algorithms biased heavily towards their training sets. That isn't to say they are overfitted, but the sheer scale/breadth gives the appearance of generalization without the substance of it.
This is very reminiscent of position-encoding neurons: https://en.wikipedia.org/wiki/Grid_cell
It is also not surprising that if you force a system to succinctly encode input-output relationships, eventually it discovers the underlying generating process or its equivalent as implied by Kolmogorov complexity theory. Language is just a convenient encoding for inputs and outputs, not fundamental. So yes it is regurgitating statistics, but statistics are non-random because of some non-trivial underlying process, always, and if you can regurgitate those statistics consistently you're guaranteed to have learned a representation of the process. There is no difference and biological systems aren't any different.
To a human, board state in chess and in duck chess is very simple. It’s chess, but with one square that’s blocked off. Similarly, even a beginner human chess player can understand the basics of duck chess strategy (block your opponent’s development in the beginning, block your opponent’s attacks with the duck to free up your other pieces, etc.).
GPT-4 fell apart, often failing to make legal moves, and never once making a strategically coherent duck placement. To me this suggests that it does not have an internal representation of the 64 squares of the board at all. Even if you set aside the strategic aspect, the only requirement for a duck move to be legal is that you place it on an empty square, which it cannot consistently do, even at the very beginning of the game (it like to place the duck on d7 as black after 1. …e5, even when its own pawn is there).
It's "internal abstract representation" isnt a representation; it's an implicit statistical distribution across historical cases.
Consider the difference between an actual model of a circle (eg., radius + geometry) and a statistical model over 1 billion circles.
In the former case a person with the actual model can say, for any circle, what it's area is. In the latter case, the further you get outside the billion samples, the worse the area will report. And even within them, it'll often be a little off.
Statistical models are just associations in cases. They're good approximations of representational models for some engineering purposes; they're often also bad and unsafe.
It exhibits internal abstract modeling.
It's a bit silly to argue against it at this time.
To produce answers with quality we see it'd have to use orders of magnitude more memory than it actually does.
It's also easy to test yourself.
Simple way is to create some role playing scenario with multiple characters when same thing is seen differently by different actors at different time and probe it with questions (ie. somebody puts X into bag labelled Y, other person doesn't see it and asking what different actors think is in the bag at specific time in the scenario etc).
Or ask for some crazy analogy.
Why am I even saying it, just ask it to give you list of examples how to probe LLM to discover if it creates abstract internal models or not - it'll surely give you a good list.