Measuring abstract reasoning in neural networks
deepmind.com
deepmind.com
That said, they are great at recognition/prediction and it is likely that you can solve some simple symbolic logic problems by recognition/prediction but only up to a degree.
Neural networks currently are very much like the sensory processing areas in the brain and sometimes they are mapped directly to actions for control of video games.
But symbolic logic as human explicitly do it, and what we consider as thinking (planning, imagining, evaluating, deciding) is not done just in the sensory areas of the brain but involves mediation by executive control centers and self-stimulation of the sensor areas in sort of a loop.
I am not sure that trying to deal with abstract reasoning in just the current sensory NN designs that we have is going to be super effective. But I guess measuring abstract reasoning will allow us to realize the current limitations and then push forward with better structures that enable it.
(Although even if the NN do not have structures to support symbolic reasoning in the way that humans do it, I guess DeepMind will just write custom code around the NN do enable NN do help with symbolic logic? Sort of how they combined NN with other search structures to create AlphaGo?
Personally I think it would be easier to combine NN with existing symbolic reasoning tools to get a better result rather than just sticking with NNs. Use NNs to recognize and evaluate patterns and feed that to the symbolic logic tools to get reasoning solutions. Much more efficient I would think and tractable. And for extra credit run NNs on the symbolic logic reasoning tools to see if you can make some "intuitive" jumps sometimes instead of pure reason.)
If, or when, it gets to human-level (for a given task) is another question. But saying that computers cannot learn how to abstract data is demonstrably false.
Vide https://distill.pub/2017/feature-visualization/ for internal representations.
Does the internal operations of "artificial" systems necessarily differ the (automatic) processes of our own mind?
And how do we know that we're _really_ reasoning?
I don't suffer under some illusion that current machine learning algorithms have reached AGI, consciousness or sentience – but we need to define these terms before arguing about them.
It seems to me that there's a general problem of definition when people talk about what systems can and can't do - and I think it all might just be a convoluted form of an "Appeal to nature" [0] in the case of these endless, unoriginal criticisms of current AI-efforts.
I don't think they have really captured that with this work. They generated some data according to some rules and fed it to a recurrent neural network, which is meant to be good at this. It seems pretty standard. I don't think it gives much insight to how "reasoning" works, or how neural networks work.
These things were originally inspired as a crude approximation to how a real brain might work. However a real brain probably has far more interconnections and recurrence. Should I be surprised if it can capture some of the real behavior?
Reasoning is not prediction learned by seeing things happen before exactly in the same way. Moving a hand to catch a ball is not reasoning, it is recognition and prediction based on doing it a bunch of times.
Reasoning to know that 4 comes after 3 after seeing that happen a lot is not reasoning, it is prediction again, and even easier than predicting the motion of a ball.
Reasoning involves finding a pattern that you haven't seen before a thousand times but you use the elements you know and look to see how they combine to fit the existing problem. Reasoning by its nature is the ability to succeed in a novel situation that requires logic and exploration in search of a problem. Reasoning is successive cycles of experimental prediction with recognition and running an evaluator on top to see which way to explore. (I think this can be coded probably in the near term, but it is more than just a single NN.)
That said, if you provide enough situations to a NN it may be able to solve a lot of "reasoning" situations via recognition and prediction, but it will sort of be stupid if anything falls out of its training data -- similar to how NNs can be fooled completely by slightly modified pictures of animals.
Unless there is of course some underlying "structure" to our phenomenological reality that affords us these insights ;)
I'd take a look at the recent CV paper on "Taskonomy" from the Stanford Graphics Lab. Instead of semantic labeling of training data by human operators. We can begin to apply learning to high resolution sensor data. Such as RGBA, motion and depth images. And uncover hidden emergent relationships inherent in their geometric and physical properties. Since these "laws" are universal. They can essentially be transferred to myriad other tasks in vision.
http://taskonomy.stanford.edu/
It is precisely this type of learning that DeepMind seeks to quantify with its tool. No one can say precisely why we as humans can look at a thing for the first time and comprehend its "thingness": its utility, its purpose, its meaning and social context. But I believe we are far closer than we imagine to robust meta-learning for intelligent machines on a grand scale!
These patterns don't translate to anything outside their domain, it would just prove IQ tests are not measuring intelligence - they measure the absorption of specific pattern data(association maps) in our brains, essentially forcing a brain to become a reactive "association-puzzle" automaton fitting symbols into RPM patterns.
My position, then, is that the system they describe cannot be said to be reasoning at all.
If your 'room' 'understands', but no part of the 'room' can be attributed with having 'intelligence', doesn't it stand to reason that the _room_ itself is the intelligent actor? I really don't see the problem with the fact that the human 'cpu' doesn't speak Chinese, the room itself 'understands' Chinese just fine.
Technically in some definitions of reasoning, such as symbolic logic solvers that already exist, those are logical reasoning optimization tools.
Not sure if complaining about it would help though.
Besides, why bother creating a whole second, parallel, terminology?