Very, very cool -- thanks for posting! Quanta is yet again proving itself to be the very best casual science publication in the English-speaking world. This is a short one, but as always, clean, original graphics + big pictures of the participants is a winning combo IMO.
Other iterative methods, like gradient descent, converge toward the true minimum at a linear rate. Newton’s method converges toward it... at an [exponential] rate...
The rate of convergence would scale with the number of derivatives used: Just as using two derivatives allowed Newton to approach the true minimum at a quadratic rate, using three derivatives enabled the researchers to approach it at a cubic rate, and so on.
I'm still confident that medium-term progress on deliberative cognition lies in the very uncool Neat/symbolic direction, but this is the first time I'm really thinking hard about what it would take to crack it the hard, more durable way (i.e. biological verisimilitude).
In terms of the paper itself (https://arxiv.org/html/2311.06374v2), this is by far the most interesting part for this novice, in 1.2:
We prove that our algorithm is well-defined in the sense that the semidefinite programs it executes are always feasible and that the next iterate is always uniquely defined...[, and that the program] has local convergence of order d. Compared to the classical Newton method, this leads to fewer calls to the Taylor expansion oracle (a common oracle in this literature...) at the price of requiring higher-order derivatives.