indeed. much better off reading this instead: http://karpathy.github.io/neuralnets/
First they come with
f(x,y) = xy
suddendly
df(x,y)/dx = (f(x+h,y)-f(x,y))/h
You could still use a deep learning framework as a black box without knowing how to differentiate a function. However, you'd have trouble following any relevant blog post / scientific paper, understanding how it works or developing any relevant new algorithm.
The article is well written, my brain just turns off if I read higher math.