330 karma · joined January 25, 2022
"That's it."
"Intriguing. I wish I had more time to discuss this matter"
"Why don't you have more time?"
"Because I must detonate in seventy-five seconds."
And having been there in the 90s seeing XML and thinking "they've tried to do everything and got into a big mess" (XSLT anyone?) its so satisfying to see something simple and flawed displace it.
I'm arguing against the 'its just a statistical model and its playing a clever trick on us' camp.
chatGPT: I'm sorry, but I'm not sure what you're trying to communicate with that sentence. Could you please rephrase or provide more context?
Instead of it being an unknown language, its English (a language you know), but every single Noun, Verb, Adjective or Preposition has been changed to Thai (a language you dont know).
The Mæw Nạ̀ng Bn the S̄eụ̄̀x.
If you had sufficient opportunity to study this pile of text, you'd begin to pick out patterns of which words appear together, and what order words often appear in. Suppose you study this giant stack of Thai text for years in isolation. After all this study, you're good enough that given a few written Thai words, you can write sequences of words that are likely to follow, given what you know of these patterns.
Right, and to get good at this task, you'd need to build models in your head. You would think to yourself, right a Mæw tends to nạ̀ng bn a S̄eụ̄̀x, and you would build up a model of the sort of things a Mæw might do, the situations it might be in. In an abstract way. As you absorbed more and more data you would adjust these abstract models to fit the evidence you had.
You dont know what a Mæw is. But if someone asks you about a Mæw, you can talk about how it relates to S̄eụ̄̀x, Plā and H̄nū. You know stuff about Mæw, but its abstract.
Right, so lets virtualise it. Actually training AIs using real cameras and real robot arms will be really slow and expensive.
So we provide a system that renders a photorealistic graphical room with teapot and robot arm, and a virtual camera inside the room is 'seeing' parts of the room and a vision model then processes what it can 'see' to try and feed info to the LLM. Likewise the LLM can make the robot arm move but its all just simulated.
Does the LLM now have a relationship with reality?
Me: There's a blue suitcase in the airport and a policeman carrying a red bag opens the blue suitcase with a sniffer dog and fireman standing nearby. The dog is worried about the suitcase but the policeman isn't. What does the fireman think is inside the red bag?
ChatGPT: As a language model AI, I do not have personal opinions or emotions. I cannot speculate what the fireman thinks is inside the red bag as it is not specified in the information given.
The other camp (that I'm in) sees that we might be onto something. We humans are obviously just more than a statistical model, but nonetheless learning words and how they fit together is a big part of who we are. With LLMs we have our first glimpse of 'emergent' behaviour from simple systems scaled massively. Whats are we if not a simple system scaled massively.
Check these links out:
Evidence that LLMs form internal models of what they learn about: https://thegradient.pub/othello/
Evidence that training LLMs on code actually made them better at complex reasoning: https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tr...
John Carmack: https://dallasinnovates.com/exclusive-qa-john-carmacks-diffe... I think that, almost certainly, the tools that we’ve got from deep learning in this last decade—we’ll be able to ride those to artificial general intelligence.
A lot of the argument comes down to semantics about knowing and thinking. "An LLM can't think and a submarine cant swim"
It 'knows' language, as in it has learnt about relationships between words (and thats really underselling it, in reality it has learnt very very subtle relationships between a great many words, and it can process words about 2000 at a time (token count etc))
BUT as you say it has no outside reference, its just a bundle of weights (those weights forming models of a sort)
BUT we provide the outside context by interacting with it. We ask it a question, it is able to provide an answer.
In any case it wont be long before someone hooks one of these up to cameras and robot arms and teaches it to make a cup of tea or whatever. A 'relationship to reality' is coming in the next few years if you think thats a critical ingredient.
Seems like its nailing it to me. You ask about a scenario and it gives an appropriate answer.
We have evidence that LLMs build models of the things they are learning about. Have a look at this paper:
Do Large Language Models learn world models or just surface statistics?
https://thegradient.pub/othello/
previously discussed https://news.ycombinator.com/item?id=34474043