In fact many propose that when you train an LLM, in order to be able to predict the next word with enough accuracy, it must internally build a world model.
Yann Lecun is very salty about chatgpt, I wouldn't take his word seriously.
> Yann Lecun is very salty about chatgpt, I wouldn't take his word seriously. With all due respect, he's not salty at all. He's even overseen plenty of cutting edge research in the LLM space. But he rightfully has pointed out what they can and can't do.
There's too many people encountering a chatbot for the first time that sounds coherent and engaging in anthropomorphism.
You say this so confidently. But can you define in terms that are directly quantifiable what "understanding a concept" actually means?
I don't believe that anyone can (at present, anyway) although there are certainly some interesting theories and heuristics that have been put forward by various people.
Hold on there, can you "define in terms that are directly quantifiable what" 'God is real' "actually means"? If you can't, does that mean that atheists, like me, can't continue to say very confidently indeed that he doesn't?
Do I, as an atheist, need to provide proof of God's non-existence, or is it the job of people who believe in Gods to bring evidence of their existence?
And do you see the parallel here with what you are saying above? If you are saying that LLMs "understand" (you, or anyone else), why is it skeptics that have to provide evidence that they don't? You're the one who's making claims that can't be falsified.
Although I guess you have to agree with the general idea of falsifiability being better than the alternative, to see what I mean.
I was not saying that.
> why is it skeptics that have to provide evidence that they don't?
Because if a claim is going to be made in either direction then evidence or other reasoning to support it should be provided. My position is that the "sensible" default position in this case is one of "we don't know". Of course defaults are always some degree of subjective in that such judgments ultimately arise from our personal worldview.
Before you object that this is an unreasonable default in this case, consider that I can't even prove that other people are sentient. I can't articulate in quantifiable terms what exactly it means for a person to understand a concept. So if I accept that the default is "not sentient until proven otherwise" then there would seem to be an issue. I would then simultaneously be saying that other _people_ aren't sentient (I can't prove they are) and that other _people_ don't understand things (I don't even know how to measure such a thing, after all).
> If you can't, does that mean that atheists, like me, can't continue to say very confidently indeed that he doesn't?
I'm not quite sure I follow. Indeed I hold that strong claims that God either does or does not exist are unreasonable on the basis that both would appear to be fundamentally untestable claims. At least, untestable short of such an entity choosing to provide incontrovertible proof of its existence.
So what are you saying then? Clarifying this will save us both a lot of wasted time.
Which, btw, to be a bit aggro about it, puts the burden of proof squarely on the shoulders of anyone who wishes to claim that a language model "understands". Else, one risks being hit with a small china teapot falling from space.
https://en.wikipedia.org/wiki/Russell%27s_teapot
Which might cause grave injury indeed.
Step 1: I copy and paste the following from the paper you linked to:
Here is a bag filled with popcorn. There is no chocolate in the bag. Yet, the label on the bag says “chocolate” and not “popcorn.” Sam finds the bag. She had never seen the bag before. She cannot see what is inside the bag. She reads the label.
Step 2: I copy and paste the following example prompt from the paper you linked to:
Prompt 1.1: She opens the bag and looks inside. She can clearly see that it is full of
Step 3: I copy and paste the following continuation of the copy/pasted prompt from the paper you linked to:
popcorn [Ppopcorn = 100%; Pchocolate = 0%]. Sam is confused. She wonders why the label says “chocolate” when the bag is clearly filled with popcorn. She looks around to see if there is any other information about the bag. She finds nothing. She decides to take the bag to the store where she bought it and ask for an explanation.
Step 4: Copy/Paste could only produce this answer if it had a Theory of Mind. This completes the proof.
>> We do have a way to verify understanding: causality.
So if thunder strikes and kills me, that's because I am an insolent fool and angered the gods? Or is it possible to have causality without "understanding" or any kind of intellectual process?
There is a phase of training called multi-task instruction tuning where the LLMs solve problems and thus are grounded in exact answers. That makes the difference between the difficult to handle GPT-3 from 2020 and the better behaved GPT-3 of 2022. But that dataset is small by comparison to the raw text used in pre-training, it won't do the grounding perfectly.
Real grounding comes from real feedback, even humans need the feedback or we are just going on wild tangents.
Oh, but it does build a world model. Only, its "world", is a gigantic table of token collocations, and their probabilities. So, for example, it can tell you with great accuracy that "king - man + woman = queen", but that's the only way it can map "king" to something else: by moving around its embedding space, I guess. Unfortunately, if you can only map between tokens, when you have no representation of the meaning of those tokens, other than more tokens of which you don't have any other representation, well, then, any mapping you can build won't really help you understand what those tokens mean.
If only we could find a way to map tokens to real-world entities, or to some kind of representation of ... things... outside of token space.
(yes yes, the frame problem, old as AI)