I think in five years, LLMs will be like expert systems: largely considered not really "AI" but sitting around in the back end of all sorts of random systems.
No. LLMs actually work. Expert systems were a flop.
I went through Stanford CS in the mid-1980s, just when it was becoming clear that expert systems were a flop. The faculty was in denial about that. It was sad to see.
We're at the beginning of LLMs, and systems which use LLMs as components. This is the fun time for the technology. Ten years out, it will be boring, like Java.
The next big thing is figuring out the best ways to couple LLMs to various sources of data, so they can find and use more information specifically relevant to the problem.
And someone has to fix the hallucination problem. We badly need systems that know what they don't know.
Expert systems actually work and, while we don’t normally even call them “expert systems” any more, are important in basically every domain – “business rules engines” are generalized expert system platforms, and are widely used in business process automation.)
Despite how well they work, and early optimism resulting from that about how much further they’d be able to go, they ran into limits; it is not implausible that the same will turn out to be true for LLMs (or even transformer-based models more generally.)
> We’re at the beginning of LLMs, and systems which use LLMs as components. This is the fun time for the technology. Ten years out, it will be boring, like Java.
…and expert systems. (And, quite possibly, by then they will have revealed their fundamental, intractable llimitations, like expert systems.)
> In fact, a good portion of the automation systems in the facilities are more than 30 years old.
Not sure what you are trying to say with that, given that the expert system hype wave started about 60 years ago and and petered out between 50-40 years ago.
The rule-based systems resurrection ~25 years ago (while seeing a much wider array of practical applications developed) had much more modest expectations, and though it was centered around expert systems as the enabling central component of broader systems, almost never used the term “expert systems”.(Though I wonder if you are thinking more about fuzzy logic, because while it wouldn’t be significant for unqualified “expert systems” as such, the timing would kind of make sense for saying something about fuzzy logic systems, whether fuzzy expert systems or neurofuzzy systems, both of which had a bit of hype cycle starting in the mid-80s which, IIRC, saw lots of attempts at industrial applications with mixed success in the 1990s.)
It's interesting how this appears to be a recurring cycle - when I attended school, it appeared that the faculty were in denial about the death of probabilistic graphical models and advanced bayesian techniques in favor of simple linear algebra with unsupervised learning. Even when taught about deep ML, there was heavy emphasis on stuff like VAE which had fun bayesian interpretations.
To put another way, I was hesitant to be as self-assuredly certain about how to define consciousness, intelligence, and sentience—and what it takes for them to emerge—as the experts who denounced Lemoine. The recent GPT breakthroughs have made me more so.
I found this recent Sabine Hossenfelder video interesting. <https://www.youtube.com/watch?v=cP5zGh2fui0>
He was frequently cited as an engineer but I don’t think he actually had a strong background in engineering but rather in philosophy.
(it's good to be a skeptic)