The advice to spend your limited time and attention on outdated approaches seems counterproductive. The things in this book aren't just old - they ended up being a dead end in research. So if it's 2023 and you have 20 hours to learn something new, you can do much better than this book.
I'm seeing rampant use of ML now for problems we already know how to solve in much simpler ways: linear control theory, bayesian statistics, Kalman filters, etc. "Oh hey, no need to study those old, dry topics. Just throw a bunch of training data at this GPU-bound black box and it will probably work."
That's right, it will probably work. Until it doesn't. And then you won't be able to debug it. More important: You won't be able to predict when the system will fail, because it's a black box. And if it's controlling a high-consequence system, when it fails people could die.
The moral is that if your problem falls into one of the already known easy-to-solve domains, you should use the old techniques. It will probably need at least 1/10^6 the CPU resources as an ML approach and you'll be able to characterize its failure regimes in advance.
> I'm seeing rampant use of ML now for problems we already know how to solve in much simpler ways: linear control theory, bayesian statistics, Kalman filters, etc.
How many of these techniques are in the book in the original post?
I'm not saying the we should throw ML at everything, I'm saying the Norvig's book isn't useful in 2023.
Yes, he would. He's written as much in the past.
Its very possible (highly likely even) that elements of gofai end up being implemented into some of the upcoming RL/GNN combo based architectures. I highly doubt that the transformer will be the end-all-be-all for generating representations. At the very least, many of those 'in-the-know' around these GNNs realize that sheaf-NNs are much more expressive and can yield far better general results if improved properly for long range dependences - perhaps with a performer or longformer -like addition.
Ultimately, some of the best researchers in the field (Velockovic is one of the best, and heavily focused on dynamic programming for example) are not just focused on the transformer or any of the hype around it right now. In order to improve, you have to look elsewhere. Understanding old methods is typically a great resource to draw that inspiration / algorithm from.
I'm not an expert in either but am confident that progress is non linear. Are there any ideas that you think are definitely bad (or even possibly good) from the lisp days?
Besides, they're not the final solution to anything.
Learning your history is the only way to avoid repeating the same mistakes.
1. Machine learning.
2. Not neural networks.
3. Not in Norvig's book.
4. Still useful and relevant.
But it seems like we finally agree on something, simpler approaches to AI that predate neural networks are still potentially useful.
Because you're not going to claim that ML is the only useful kind of AI, are you?
Practically, which other book / ressource should someone with little time check out first ?
My version is from when I went to school 20 years ago. I assume it's been greatly updated over the past couple of decades. I wonder if it's worth taking a spin through the new edition.
http://aima.cs.berkeley.edu/contents.html
Chapters 19 on are going to be the biggest additions from the earlier editions.
(Interesting that AI is finally catching up with javascript frameworks.)
My personnal goal is to find some time to dig into https://course.fast.ai/ , assuming it's not terribly outdated, either.
The world isn't moving that fast. Transformers and LLMs are built on neural networks and lots of data and fast computers. You could jump straight to that point, but even the course you've pointed to starts off with more foundational ANN topics before getting to transformers. Much of which is at least in the TOC for the current edition of AIMA. Ought to be complementary texts.
Also, only fools ignore history, "classical" AI and topics also covered in the book are still applicable. ANNs aren't going to solve all the world's problems. Other techniques that fall under the category of "AI" are still applicable and very effective for a large number of real-world problems (and much more efficient than LLMs).