Hacker News Reader

>also play a huge role in automatic differentiation

Could you extend more on how GA plays a role in this? I wish to learn more but don't know where to start. Any leads would be appreciated greatly.

View on HN

In the video they present it as an algebraic framework where you can "add" to a number system elements x such that x^2 = -1, x^2 = 0, or x^2 = 1. Adding the element x^2 = -1 to the real numbers gives you the complex numbers, with x = i. Adding the element x^2 = 0 to the real numbers gives you the dual numbers with x = epsilon, which is what can be used for automatic differentiation. The case of x^2 = 1 is more complicated.

Don't know of a good single resource that is comprehensive. A high level overview of autodiff approaches can be found here: https://www.robots.ox.ac.uk/~tvg/publications/talks/autodiff...

Some example implementations based on dual numbers: 1. http://www.ceres-solver.org/automatic_derivatives.html 2. http://www.imm.dtu.dk/~kajm/FADBAD/

Newer ML frameworks do source to source transformations, which allows calculating the derivative without changing the function signature, but the concepts used remain the same.

Reply on news.ycombinator.com