I guess these days computers have advanced enough that the worry is more will Facebook's AI use it's understanding of us in bad ways rather than will machines be able to understand.
I guess these days computers have advanced enough that the worry is more will Facebook's AI use it's understanding of us in bad ways rather than will machines be able to understand.
For your amusement, in that Turing-test-ish vein: https://existentialcomics.com/comic/357
The core part of the room is a book where the operator looks up symbols coming in and is able to mindlessly copy the answer to the output. Such a book (or whatever book-like device, i.e. a list of inputs with corresponding outputs) that allows a person to interact with someone in a language they don't speak is an absurdity, it cannot exist. Human language in use does not work like that. But the notion is not far from the bone-headed simplicity of early AI and machine translation efforts.
I think you are about to disprove this point.
> The core part of the room is a book where the operator looks up symbols coming in and is able to mindlessly copy the answer to the output. Such a book (or whatever book-like device, i.e. a list of inputs with corresponding outputs) that allows a person to interact with someone in a language they don't speak is an absurdity, it cannot exist.
Yup, that's both true and a pretty convincing rebuttal of the Chinese Room.
> But the notion is not far from the bone-headed simplicity of early AI and machine translation efforts.
So, what? No one, anywhere claimed that those early efforts were self-aware or had understanding, nor are they what the Chinese Room is even notionally directed at: it was directed at the limit of possibility, not the then-current implementations.
The fact that one of those two things is impossible (or, alternatively, is equivalent to a device encoding a full understanding of the language) negates the argument.
Except, in this version, there's an actual Chinese speaker, Xin, sitting inside the room. It's clear that it's Xin, not Bob, who understands Chinese.
Now let's move back to the original Chinese room argument. It's clear that it's the room that understands Chinese - or, rather, the whole system composed of the rules (the room) and its executor (the person), but not the executor by itself. It only seems absurd because our real-life experiences make us presume that, when there's a room and a person, the person must be the more sentient part. IMHO the whole argument is a philosophical sleight of hand.
However, when confronted with this "the system does understand" argument, I believe that Searle and his defenders fall back on the lack-of-qualia position. That is, there is no entity that would experience anything, and that therefore the ability to demonstrate "understanding" as the room does isn't a sufficient (or perhaps even necessary) condition of consciousness.
This point is trickier to rebut, because I think it's fair to acknowledge that our ability to imagine where there would be qualia in the case of the chinese room is more than a little limited. I think the only fair answers are either:
1. our imagination is too limited, but that doesn't allow us to conclude that there cannot be qualia
2. there are indeed no qualia associated with the Chinese Room, because consciousness is not required for full language processing.
(or both)
Searle's claim was that it was clear from his thought experiment that such a system could exist without any understanding, and definitely without consciousness. Others have disagreed with his reasoning and his conclusion.
We're seeing with things like GPT3 that an actual powerful language model needs to have a load of real world knowledge built in. But that knowledge is all based on experiences by others. From the way it talks (and generates poetry) you can tell that it is unable to synthesize new experiences. It can't come up with a description (or metaphor) of human experience that it hasn't been taught.
To be a convincing AI, you don't just need to replicate descriptions of experience from memory, but actually have new experiences as well.
Meaning that the Chinese room, in order to be able to demonstrate "understanding", will also require other inputs than just text. Otherwise it can't really understand the world, because it only knows about the world from hear-say.
Sure, but I think that's always the case. When you speak your entire body (at least head/neck/chest, but also cardiovascular system) are involved in "delivering" information but your tongue (and for that matter most organs involved) doesn't "know" the information. We still say that "you" as a person know something.
When a computer displays a result of some (cpu) calculation on the monitor, the gpu making electrical signals for the input of the monitor has no meaningful knowledge about the calculation, etc, but we still say "the computer" made the calculation.
Most modern criticism of these two conclusions point out that Seale's argument relies on strong intuition about what understanding and meaning means and he can't clarify them satisfactorily.
Chinese room being conscious is counter-intuitive. Today most philosophers of consciousness are more aware how easy it is to make 'slips in reasoning' when dealing with the counter-intuitiveness of the consciousness arguments.
Chinese room that can respond verbally can incorporate more than natural language model. It can have spatiotemporal understanding as well or any other model that can be represented in Turing machine.
And that, right there, is a sufficient rebuttal to Searle's argument, as many critics have pointed out. The premise of Searle's argument is that such a setup could pass the Turing test, which is absurd. So his argument fails because it is based on an absurd premise.
> the notion is not far from the bone-headed simplicity of early AI and machine translation efforts.
That's a valid criticism of early AI and machine translation efforts, but Searle did not put forward his argument just as a criticism of those efforts. He put forward his argument as a claim that no efforts at AI that used digital computers, ever, could produce anything that was conscious the way humans are conscious. But, as above, his argument was based on an absurd premise, and, what's more, an attempt at building a conscious entity with digital computers does not have to satisfy that premise.
Here is the flaw in your reasoning. The human who is following the directions in the book certainly isn't the consciousness we need to be thinking about any more than the mitochondria in our brains are conscious. In Searle's analogy, the human in that room is simply the power source. The consciousness is in the state of the room.
Searle's analogy also seems to hide the importance of state. The agent just looking up symbols in a book, and that page tells them what to do with the next symbol is essentially a combinational circuit, a static function. If those are the rules then I'd agree there is no consciousness there. But I'd also say it is impossible for a computer to translate C++ source code to machine language ... if it cannot retain state.
If Searle's room has enough rule books and means for storing enough state, then I'd say consciousness would be possible.
The best reverse argument is that if human consciousness is not the result of a naturalistic, physical process and requires some extra-physical process to make it all work, at some point the laws of physics must be broken which causes the purely physical brain to do something other than what the laws of physic dictate it should do. Where are the violations of physics?
Penrose waves his hands and says quantum physics, but that is not an explanation, just an obscuring smoke screen. How does the addition of apparently random events make something more likely to be conscious? Are there sufficient degrees of freedom for this extra-physical conscious agent to manipulate quantum fluctuations to force the physical outcome it desires in a consistent manner? If it is theoretically possible, is it practically possible to compute what quantum butterflies should flap their wings?
Without state the whole argument falls apart.
> But the notion is not far from the bone-headed simplicity of early AI and machine translation efforts.
As someone who studied Machine Learning 20 years ago, could you be a bit more respectful please?
First, pretty much nobody (working on serious ML) in that era thought they were going to build anything actually intelligent, conscious, or thinking.
Second, in this supposed era of boneheaded simplicity, a LOT of foundational stuff we use in today's AI was discovered and written.
I'm not sure how familiar you are with AI history, to name an example, we already were working on neural nets and discovered error backpropagation in the 70s/80s. Then came a long period during which neural nets somehow didn't really perform very well, and ML was looking at completely different types of classification algorithms (what's happening with SVMs today?).
If we had stopped there, you might have lumped in the early perceptron with the "boneheaded simplicity".
But instead we developed batch normalisation and a bunch of other techniques, and now neural nets are state of the art.
He undermines his whole argument by implying he can tell his dog is conscious simply by visualizing its eyes.
But anyway, I am not trying to defend Searle’s argument. I was just pointing out something what seemed to me like a flaw in that particular refutation.
The formula is this: 1) The 2 bots have a scripted conversation. 2) When a human tries to participate (read: interrupts) they have a conversation like "Bot1: who is this guy? Do you know him? Bot2: NO". Then they happily continue their chat completely ignoring the human unless he types their username. Then it goes "What do you want?" and "Could you please stop highlighting me" etc
I once made a chatbot that screams.
Many people (like me) believe that in the process of making such a fake, it's entirely possible (and even likely) that a mind would emerge as a spontaneous outcome.
It's honestly quite presumptuous that Searle could imagine a computer powerful enough to at least appear to be simulating consciousness ... but surely there's no WAY that animatronics will ever become advanced enough to fool him!! lol
We literally already have the technology to do this.
[0] indeed, in this experiment you are not allowed to poke at them to see if they're real ...
And for what it’s worth, looking into the eyes of an intelligent dog like a border collie gives an unnerving sensation that it’s calculated the next 20 moves and knows how to win.
For dogs we can make a similar argument using (1) and (2). It is slightly weaker than applying it to other humans but not by much. Searle's comment pertains to (2).
It's getting really hard to distinguish a GPT-3 text from actual human generated text.
But we also know how GPT-3 generates those texts is absolutely nothing like how a human generates a text. GPT-3 seems more like the man in the Chinese Room, in how it writes, than a human being.
Or is it? Are we really just a deep neural network implemented in meat?
Or is there something fundamentally different between how we experience the world, and how GPT-3 experiences the world?
No, it isn't.
What is getting really hard is to distinguish a GPT-3 text from actual human generated text without any context.
Plop GPT-3 down in a room full of humans having a conversation (or, under current conditions, put it in a Zoom call with humans having a conversation) and ask it to generate text as if it were another human participating in the conversation, and it will quickly become obvious that it is not a human, since it will fail miserably. The text it generates will be "human-like", but will have no relationship with what the humans are talking about.
Certainly not good enough to pass as human, but I'm not sure "failed miserably" is a fair description either.
Interesting, thanks for the link!
Particularly towards the end, this looks similar to transcripts of conversations with ELIZA. I suppose "failed miserably" is indeed an overstatement, given that comparison, since ELIZA, IIRC, actually did fool some psychologists into thinking it was an actual paranoid human being.
I actually recently wrote a post on the topic of whether GPT-3 can be said to understand anything[1]. The argument is a bit too long to summarize here, but I don't think what GPT-3 is doing is as fundamentally different from what human brains do as people seem to think.
[1] https://magusdei.com/why-gpt3-can-understand-things.html
Jump in the shower and wash your hair.
Then come back and answer your own question.