Can someone provide insight into why there's so much insistence from business that magic happens at scale with LLMs? We're a long way from AGI.
The lack of meaningful details in these announcements makes me pessimistic.
The lack of meaningful details in these announcements makes me pessimistic.
It's already happening. See latest Google layoffs. They are automating a lot of things. Most people don't realize it, but the change is going to be dramatic.
> We're a long way from AGI
This is a big question, what is AGI? LLMs are quite generic and 'intelligent'. Not human-like, but. Next is going to be incremental evolution. Till we find other, non-verbal, ways of 'thinking' and put them together. That's going to be a breakthrough. Interesting, terminator-like embodiment isn't a requirement for AGI, nor is stable 'personality'.
Intelligence is not about parroting an answer you've seen before. It's about using your environment to gain an evolutionary advantage.
I don't think that's right either (it sounds like a description of adaptation), and I don't think your description of LLMs is fair, even though I'm fairly sure they're not AGI and won't scale to AGI.
Intelligence is more like the ability to generalize skills, applying knowledge gained in one scenario to another scenario.
the ability to apply knowledge to manipulate one's environment or to think abstractly as measured by objective criteria (such as tests)
Manipulating the environment afaik is to increase your chances of survival.
Hmm.. you are talking about LLMs.. They are the most generic thing we have right now (Jan 2024) LLMs have limitations, like learning on the fly isn't their strong side. But the same with brain, it consists of limited components, that's only together they work well. LLMs can be a part of the solution, if we can't find something better.
That's more like robotics. Except for evolution part. Does AGI require breeding? Software can easily multiply itself. That's hardware is the problem then.
Intelligence emerges as you are trying to survive longer. Reproduction is the ultimate way of cheating death.
Environment does not necessarily needs to mean physical environment, but until the "AI" does not recognize that it is in danger of nonexisting and starts to behave in ways to avoid it it cannot, imho, make the leap to AGI and it's just a really sophisticated tool.
That's not a given - I don't even know if it's true. The longest-lived species are not very intelligent, relative to humans. Intelligence is a tool that may or may not evolve in organic species. Frankly, that has very little to do with defining what artificial intelligence is.
What exactly did they automate?
Their expenses haha
Some more pressing questions are:
* What new capabilities emerge as the models get better and better at predicting (i.e. loss goes down)?
* How much will it cost to train increasingly large models? And to run inference on them?
* How difficult will it be to find or generate more and more high quality data?
what are they then? I thought everyone was firmly in the "let's train with more data and see what happens" camp
How loss translates into higher-level capabilities is anyone's guess.