http://norvig.com/Lisp-retro.html
The next to last section in this, What Lessons are in PAIP? comprises 52 numbered sentences of advice, cross-referenced to fuller explanations in the book. Many (most?) of them apply to any programming or software engineering, not specifically AI or Lisp.
https://en.wikipedia.org/wiki/Macsyma
Similar to the way Python today is a high level language used for numerical computing today. Lisp is certainly more than capable of this role, but the fallout from the AI winter killed Lisp's reputation and popularity, leading to other languages filling those niches.
That said, seems most of what is important in today's code is the ability to drive a gpu. Don't know why that couldn't be done from a lisp.
At least my take on it is: what if you simulate a domain (physics being the most tractable) then can you train using differentials of the model itself and not just differentiating the internal neuron estimators?
Lots of open questions... like what's the driving force that asks for an effect that calls for differentiating the model? We kind of expect the training process to just recreate everything internally.
Now it has me thinking about "coprocessors"... a lot of the domain-specific math is given to the NN as input features, but what if the NN could select inputs to those algorithm? That would give the NN the ability to speculate. Of course you'd need to also differentiate those algorithms.
Now that I'm thinking about it, I guess this is just another activation function, just one that is domain-specific. Are there approaches that use eclectic activation functions? Like in each layer 1-10 is one activation function, 11-20 is another, and then you finish off with tanh or relu or some other genetic activation function.