I don’t think it’s going to be a “winter”, but there are definitely some bubbles to burst. Especially when LLMs become half-assed products and the general public’s heightened expectations are not met.
The definition of it isn't clear but from what I gather it's basically an aggregate of emergent capabilities that work together to produce a singularity.
Maybe with enough resources it's possible but I highly doubt it'll be economically feasible given how much has gone into it so far and how far we really are away from something like that with current models.
In an age of such hopelessness about the future, this looks a lot like an emotional crutch wrapped in the veil of rationality - just the thing an anxious materialist needs to make sense of the world.
Like many cults and religions it mistakes the plausible for possible and possible for probable.
The problem with religious beliefs like these is that they don't just disappear with evidence or sufficient reasoning.
I don't think that particular bubble is bursting anytime soon.
It's useful but basically every method of quality control requires a human.
I've found that components of general intelligence specialized beyond human capability are much more useful than a model that can mimic a human.
I think an LLM is just trying to do too much at once, all of the individual NLP algorithms most of them are made of are very useful to us, but an LLM is just not specialized enough to be any more useful than a human without specialization.
Which isn't to say they're _useless_, but obviously not as useful as a specialist (in special contexts, denoted by whatever kind of specialist they are)
ETA: as an aside, I'd like to contextualize my presumption that AGI is about AI singularity with the fact that Sam Altman casually stated that he doesn't care if it takes $50 billion to reach AGI.
In the real world, with 50 billion dollars, you can do something much more useful than trying to build a product that's basically contradictory by definition.
An AGI is (presumably) a general intelligence model but it's implicitly touted as being extremely useful for specialized tasks (because, humans can specialize), but once you specialize, I would argue your general intelligence tends to weaken. (For example I wouldn't expect a Harvard PhD to be 100% up to date with modern slang terms, but I'd be shocked if I went to a local bar and met someone who didn't know what rizz means).
This is basically just trying to squeeze two opposite ends of a spectrum together, which sounds kind of like a singularity to me.
I get that. I guess my point is this already seems to exist. We could combine AI with machinery to replace almost everything humans can do already, someone just has to build for that solution (e.g. train some models).
AGI just sounds like a sort of automation of that process. And I don't think a bigger LLM will accomplish that task. I think more developers will.
Which I wager would be cheaper and arguably more fortuitous to the human race than $50 billion thrown into one pot
But yeah developers are needed, a bigger LLM won't fix everything.
The money's a funny one. Global GDP is about $85,000 bn/yr so if someone can spend $50bn on getting AGI and taking it over it's a bargain. But if you spend $50bn and just get a loss making chatbot then less so.
Also, I still think you can probably build something (or rather, many, many somethings) with existing tooling to accomplish exactly that.
> A bigger LLM won't fix everything
I'm not sure if there's a camp that says it probably won't fix anything, but I'm in that camp if it exists.
If you think about how humans actually work, I think a basic, non AGI LLM routing information to different agents/models is closer to how most humans behave (when productivity is their goal).
E.g. a person's behavior is driven almost entirely by the current context they are in most of the time.
It's not that our minds become overexcited by loads of previous information and we magically are able to do other specialized tasks, we decide based on context what specialty in our toolset best fits the scenario.
> The money's a funny one. Global GDP is about $85,000 bn/yr so if someone can spend $50bn on getting AGI and taking it over it's a bargain. But if you spend $50bn and just get a loss making chatbot then less so.
If that's true then the same could be said of just dumping $50 billion into grants/research/funding for education around AI so that developers worldwide have an easier time developing AI enabled technologies and services.
At least with that plan, there is extremely little risk of creating nothing more than a chatbot (and extremely low risk of tech companies monopolizing labor the same way they try and monopolize everything else; I don't have much faith that if a few companies automate all or most labor that they'll redistribute wealth)