I'd just like to interject for moment. What you're refering to as Deep Learning, is in fact, non-convex optimization, or as I've recently taken to calling it, generalized function approximation. Deep Learning is not a technology unto itself, but rather a specific type of mathematical optimization that has been known about for decades, is based on stochastic gradient descent, and only now is being featured more prominently due to enhaced processing power.

Many computer users run some form of mathematical optimization everyday, without realizing it. Through a peculiar turn of events, the optimizations which are widely used today are often called Machine Learning, and many of its users are not aware that it is basically non-convex optimization, or high dimensional function approximation.

There really is Deep Learning, and these people are using it, but it is just a part of the optimization mathematics they use. Deep Learning is the kernel: the program in the system that allocates the machine's resources to the other programs that you run. The kernel is an essential part of an operating system, but useless by itself; it can only function in the context of a complete rigorous mathematical framework. Deep Learning is normally used in combination with other ideas in classical mathematical optimization: the whole system is basically stochastic gradient descent with GPUs added, or large-scale numerical optimization. All the so-called Deep Learning algorithms are really algorithms of generalized function approximation!