> Deep learning entered the sphere of mainstream awareness less than a decade ago, but the science is ancient. This 1986 paper in Nature contains almost all of the core technical innovations that make it work well. (The term deep learning hadn’t been coined yet and neural networks was used instead).
> However, the dataset sizes and compute resources that were available in the 1980s weren’t enough to demonstrate the effectiveness of deep learning, and other machine learning techniques like support vector machines took center stage.
I think there are two glaring errors here. First, multilayer error backpropagation wasn't developed in 1986. It was developed in the early 1970s, but was quashed by the elimination of funding after the Perceptrons papers and book. It was rediscovered several times but didn't take hold until the PDP group (Rummelhart et al) pushed things through with enough gravitas.
Second, and this is the big one: there were plenty of reasonably large datasets, and frankly plenty of sufficiently large compute resources in the late 1980s and certainly the 1990s to do this research. That wasn't the problem. The problem was that neural networks were largely restricted to two layers due to the vanishing gradient problem, so they had to be fat rather than deep, which wasn't good. That's the whole point of the name deep learning -- by overcoming issues which prevented deep networks, researchers were able to do the stuff they can now. That didn't come until later.