To be fair, this is machine learning we are talking about, not algebraic topology. The experts in ML are still proud of the fact that they figured out the chain rule...
I assume you're talking about using backpropagation with gradient descent. Backpropagation itself isn't all that interesting. The interesting part is that it works for practical problems and doesn't get stuck in shallow local minima.