Neural networks are very non-linear. The blessing and curse of non-linearity is its usefulness while escaping what mathematical logic inherently is able to model. So yes, I strongly assume there will never be a mathematically sound theory that predicts deep learning in any meaningful way. Same problem with everything non-linear. From two-body to chaotic systems, biological interactions to the stock market. All just wishful thinking until Mathematicians give up and instead continue to play around in their well-defined esoteric spaces.