When'll we see theoretical and mathematical foundation for deep learning?
quora.com
quora.com
That is a far cry from WHY a DNN-based program works. That is like asking WHY I used a Fourier Transform, not why a Fourier Transform works. We will soon reach a situation where a self-driving car decides to suddenly slam into a parked car. Probably because the heat waves rising from the road made the telephone pole look like it "moved" and was "alive" so the car decided to avoid it, choosing the "inanimate" parked car instead. Oh, the joys of debugging THAT program... the weight on the third layer node 1273134 should be .077651 not .077650
There is a government Broad Agency Announcement (BAA) (http://www.darpa.mil/attachments/DARPA-BAA-16-53.pdf) that hopes to address this issue. It will be interesting to see if that research addresses the first idea or the second.