Hacker News Reader

Comment by crubier | Hacker News Reader

This video is the best thing I watched in the past few years. Last time I loved something that much was Geoffrey Hintons course about deep learning back in 2015.

It is eye opening, just like in the beginning of mainstream deep learning, it made me wonder “why didn’t we discover this before?” It elegantly solves so many annoyances about geometry, maths and physics.

I’ve been evangelizing Geometric Algebra like crazy since I found this video, including in hacker news comments.

View on HN

Agreed. Besides Geometric Algebra, dual numbers also play a huge role in automatic differentiation -- the core building block of modern machine learning frameworks.

I really like the analogy in this talk about how Al-Khwarizmi's six quadratic equations simplify to just one, once we learn about negative numbers and zero.

In a lot of ways, geometric algebra (and dual numbers) are our discovery of "negative numbers and zero", but for the 21st century.

>also play a huge role in automatic differentiation

Could you extend more on how GA plays a role in this? I wish to learn more but don't know where to start. Any leads would be appreciated greatly.

In the video they present it as an algebraic framework where you can "add" to a number system elements x such that x^2 = -1, x^2 = 0, or x^2 = 1. Adding the element x^2 = -1 to the real numbers gives you the complex numbers, with x = i. Adding the element x^2 = 0 to the real numbers gives you the dual numbers with x = epsilon, which is what can be used for automatic differentiation. The case of x^2 = 1 is more complicated.

Don't know of a good single resource that is comprehensive. A high level overview of autodiff approaches can be found here: https://www.robots.ox.ac.uk/~tvg/publications/talks/autodiff...

Some example implementations based on dual numbers: 1. http://www.ceres-solver.org/automatic_derivatives.html 2. http://www.imm.dtu.dk/~kajm/FADBAD/

Newer ML frameworks do source to source transformations, which allows calculating the derivative without changing the function signature, but the concepts used remain the same.

Reply on news.ycombinator.com