- Failure to distinguish b/w narrow and human-level AI
- Zero mention of attention/transformer models
- Zero mention of BERT, GPT, let alone GPT-3
Note-to-self: ignore and file away in the gary-marcus box.
- Failure to distinguish b/w narrow and human-level AI
- Zero mention of attention/transformer models
- Zero mention of BERT, GPT, let alone GPT-3
Note-to-self: ignore and file away in the gary-marcus box.
1) any computational AI can be represented as a finite state machine or FSM (author calls it a finite state automata. Same thing.)
2) when said computational AI performs an "act of cognition" it (as a FSM) will iterate through a defined series of states, based on a defined series of inputs
3) It is possible to build a simpler FSM composed of a counter and a lookup table that would take the same series of inputs + the counter as an input, and produce the same state/output as the original computational AI
4) since the response to stimulus is identical, the 2 finite state machines are equivalent.
5) if the state machines are equivalent, they must be equivalently conscious
6) the counter+look-up table is obviously not conscious ("reducto ad absurdum")
7) from (6) and (5) no computational AI can be conscious
To me, this argument fails in the following manner; the only way to actually construct the "simpler" finite state machine in step (3) above is to actually let the computational AI react with the world first and record its combinations of input and state. There is no way to predict what series of states an arbitrary FSM will go through in response to a particular series of inputs without actually running it. That would be equivalent to solving the halting problem. Any program can be encoded as an FSM. If you could predict the state sequence of such an FSM, you could tell whether the FSM would enter the 'halt' state.
IMO this is analogous to arguing that:
1) the animatronic band at Chuck E. Cheese could be programmed to play identical music to that which has been (previously) performed by a human band (and recorded in perfect detail).
2) because they produce identical outputs the 2 bands are equivalent
3) if they are equivalent, they must equally be said to create original music
4) the animatronic band obviously doesn't create original music
5) from (3) and (4) no band can create original music
He also elides any discussion of whether or not actual human intelligence manages to avoid the failure modes he uses to conclude that neural networks are not intelligent - e.g. he mentions adversarial examples fooling visual classifier networks without mentioning that "optical illusions" exist and people will reliably misperceive certain images in certain ways too.
I actually agree that neural nets as they currently exist are aggressively stupid, but the author concludes way too much.
TL;DR, author starts from a premise that there is something uniquely special about human consciousness that machines can't duplicate, and reaches the conclusion that there is something uniquely special about human consciousness that machines can't duplicate.
Best not to ignore.
We're seeing AIs that can kind of hide their lack of any kind of understanding by means of more and more sophisticated behavior. But unless understanding = "not understanding, but in a really sophisticated way", that road isn't going to get us to AGI, ever.
Personally, my bet is that understanding != "sophisticated non-understanding". I can't prove it, of course, because I don't know what understanding is either...
[Edit: I suppose the other alternative is that understanding is real, but AGI doesn't actually require understanding. That seems improbable to me, but it is at least theoretically possible.]
The missing piece for real understanding is embodiment - we can act on our environment while GPT-3 like models can only see a fixed training set. This means an AI could construct its own hypothesis and test it if it were embodied, but can't do that otherwise. Understanding comes from playing 'the same game' with us: sharing the same environment and having aligned rewards.
Imagine if a scientist were kept locked up all her life and only had access to a video feed showing the world. Would she gain anything by coming out of the cave and actually seeing and interacting with the world?
It was only believed that Go was impervious to the brute-force search strategies employed by the most successful chess programs. The alternative was never "true understanding", but rather believed to be the application of knowledge engineering, expert systems, etc, none of which could be equivocated with "true understanding".
Regarding poetry, the counter argument is that a modern neural network can't generate poetry any more than a Xerox machine can generate poetry. IOW, it can only replicate styles, not invent new ones. Though, that's often good enough as far as the vast majority of readers are concerned.
You assume, but are you sure...?
https://www.theregister.com/2013/08/06/xerox_copier_flaw_mea...
Did somebody create a program that can write poetry without first consuming basically everything humans have ever written? Or even after consuming roughly what a well-read human might have read?
I think it's reasonable to allow for the fact that humans must have a certain amount of information encoded by evolution, but I can't believe it's either qualitatively or quantitatively comparable to a database of all the text on the internet.