If you want to learn how to build autodiff from scratch in python, see autodidax: https://jax.readthedocs.io/en/latest/autodidax.html
It's light on math and heavy on simplicity. I've been studying it for about 9 months and I still learn new things.
It's light on math and heavy on simplicity. I've been studying it for about 9 months and I still learn new things.
Here's the org-mode export of the file, with all TeX rendered: https://samritchie.io/dual-numbers-and-automatic-differentia...
I worked on this with Gerald Sussman, and it has the somewhat unique feature of being able to take derivatives of higher-order functions. The system is also extensible, so this feature could have been installed outside of the core library, and in fact would need to be to support, say, functions that returns JS futures, scala functions etc.
Enjoy!