> One of the most celebrated successes of Deep Learning is image labeling, using CNNs, Convolutional Neural Networks, but the very essence of CNNs is that the front end of the network is designed by humans to manage translational invariance, the idea that objects can appear anywhere in the frame. To have a Deep Learning network also have to learn that seems pedantic to the extreme, and will drive up the computational costs of the learning by many orders of magnitude.
Thanks for the link, this is a pretty strong rebuttal.
That said...
> Massive data sets are not at all what humans need to learn things so something is missing. Today’s data sets can have billions of examples, where a human may only require a handful to learn the same thing.
I've come across this idea before, the notion that we should be able to create an AI that can learn something from just a handful of examples because humans can do that. This assumes some rough level of equality in lack of related knowledge / capability between a tabula rasa AI system and a human adult. But a human adult has been awake and continuously learning & testing conscious and unconscious predictions about everything they interact with for at least 18 years (1 year = 31536000 seconds * 2/3 (accounting for sleep) = 21024000 seconds awake per year, * 18 years = 378 432 000 seconds). In other words, a human adult is already a highly trained system. And even very young human children have been awake and learning for quite a long time compared to anything we might imagine to be a tabula rasa system.
So I don't think that the human capacity for learning supports the idea that a tabula rasa AI system should be able to learn any particular thing with just a handful of examples.