Toward AGI – What Is Missing?
mark-riedl.medium.com
mark-riedl.medium.com
This is not a utopia, but a juvenile and unthinking definition of utopia, or a dystopia in disguise. Most people thrive on some sort of challenge to at least do basic tasks for themselves. A world where AGI fulfils every need is more like being in a senior's home where everyone does everything for you.
It sounds good at first but is actually a nightmare where your autonomy is stripped by the lack of goals to work towards.
Immense surplus and easy access to everything all the time means human beings will no longer need each other much either, so it means a world full of people uncaring towards each other. Fulfilling any want without hard work to get it means becoming effectively like a bacteria, consuming every resource in sight, transforming us into even more of a plague than we already are.
I certainly would not want to live in such a world.
It's just a depressing viewpoint imo
And now we're trying to use AI to become even more resource-intensive, burning up endless energy to be even more powerful consumers...give me a break. It's sick. As individuals there's nothing seriously wrong with most people but en masse with modern technology, that is exactly what we are, a plague.
Who has that as a goal?
One thing that the market will reward handsomely is bringing costs down. When costs go down, people consume more. If you're 40 or older, think about how many computers your family owned in your house growing up. We owned some hand-me-downs, and purchased 1 computer in my first 15 years. We bought about 4 in the next 3 years because they were advancing rapidly, but more importantly, because they had become much cheaper than they had been 5-10 years before. Today, many families might have a laptop for each person, replaced every 3-4 years.
One of the first things an AGI would be put to work doing is manufacturing, as we're already finding that it's hard to get people to do these stressful manufacturing jobs except in places where they're really desperate. Witness the moving of jobs from US -> China -> Southeast Asia.
There is no good without humans. Any philosophy that asserts otherwise fails Camus's fundamental question.
It's not just us. Earth might be fine but a great many species won't
>Plants and animals kill each other in a ruthless cycle of violence, too. We're just really good at it.
It's not at all comparable lol. Not in scale or function.
I actually think that the disadvantages of having few goals would become immediately obvious to almost everybody, and a whole new type of skill / creative / athletic based "economy" might emerge. But with lower stress and lower stakes
As opposed to what? Trust your hunch and halt technological progress. Wrap it up folks, we're all done with R&D now.
Your concern about purpose strikes me as odd. When people derive purpose from their work, it's often because they feel they're playing a role in a community. To deprive people of purpose would require that we make communities fully automated and impersonal. I don't think anyone wants or is proposing that.
Absent a requirement to spend one's time earning money, some people would probably degenerate into sloths, but plenty of people are otherwise motivated by self-expression, socialization, competition, etc. People would still interact and do things because people like interacting and doing things.
It is really not about wanting or proposing anything. It is about the effect of technology and what it will mean. Don't you think that communities are already becoming a bit more impersonal and automated? With ChatAI replacing customer service, self-checkout aisles, online dating, etc...
It seems to me that communities are indeed becoming more and more impersonal, especially if you compare them to communities thousands of years ago.
Definitely these interactions have become less personal, but aside from online dating, which is its own can of worms, the interactions are all transactional. I actually much prefer to interface with a predictable machine when purchasing things or making an RMA request or whatever. I don't want to stop to chit chat or worry about maintaining a relationship with the village cobbler's son to ensure he'll do his best fixing up my field boots.
The ongoing disentanglement of socialization and daily necessities is liberating. It allows one to associate with others by choice. I'm not a huge fan of the tools we have for this sort of voluntary community building at present, but I see no reason why they couldn't be improved, especially if people were able to spend their time and skill doing so free of profit motives.
Or like being mega-wealthy.
There are plenty of folks at senior homes enjoying the ability to try new things they didn't previously have time for.
You're always free to set your own goals, and pursue them as you see fit. Hell, look at the leisure rich: They're liberated from work already. Their lives don't seem particularly miserable.
My post was more subtle than that anyhow. I don't advocate work for work's sake, especially working FOR some worthless corporation. But swamping the world with AI-created things will also make most leisurely creative activities difficult for humans to do. So even purposes you create will be difficult to share with other people.
AI also floods the world with a surplus of random content, which will make people less likely to appreciate what you share with them. Perhaps you are old enough to have developed a good network of people, which gives you some shielding against it, but for babies born today, they will have a much harder time developing such a network and they will not grow up right.
I'm not sure what you mean by "needing other people." If you're talking about emotional need, I don't think AI really substitutes. If you mean pragmatic needs, then that's not what real friendships are based on.
I recently saw a “modern art” exhibition and one of the things on display was quite literally a black square in a white wall. That’s it. AI could have produced that long ago. But presumably the artist, and at least some of the people viewing it, got something out of this as a means of expressing “something” (I don’t know what, I clearly didn’t “get it”). If AI liberates artists from having to make money from their art, then they can focus on expression and “art for arts sake” without having to worry about appealing to whoever is likely to pay them.
Sure, the AI could create the same art, but it could create that black square too, long ago. That didn’t stop this particular artist.
[1] I don’t think it will, automation has already improved productivity and production a thousand fold over the past 100 years, yet we still have to work just as hard while a handful of rich people are richer than ever before, with more personal wealth than sone nations. More likely, AI will make a handful of rich people even richer still leaving the rest of us in the dust, probably being put to work on meaningless grunt work — don’t want to waste valuable AI time on flipping burgers now do we!
It is not just capitalism that demands hard work but even some challenges to capitalism. I am again reminded of a bit of 1960s Maoist literature translated from Chinese into English that I once found in a Chinatown bookshop: in discussing the Peking Man fossil, it claimed that what set Man apart from his ape forebears was that he had the capacity to labor. Those who did not wish to dedicate their lives to labor, the book claimed, were reverting to primitive ways.
And can you really say that you would have appreciated getting to where you are, and FINDING your purpose if your WHOLE life were provided for by AI all along the way??? Sounds like you might have actually worked a little to get where you were. Imagine a life where you didn't have to do any of that....would you have the same WISDOM to appreciate where you are now??
But somehow people don't mind...
It is much easier now to cheat at chess than it ever was, and it even happens at the grandmaster level. In some ways, the most recent controversy with Magnus Carlsen is due to computers and cheating. Moreover, many people cheat at online chess.
Finally, chess is becoming less interesting for sure, now that computers have introduced such a high level of mechanization in the study of it.
SURE, people STILL enjoy chess, but one could CERTAINLY argue that computers, Deep Blue, and Alpha Zero have made chess less interesting due to cheating and over-analyzing. That is why some people have even got to playing random chess (random starting position), because the overemphasis on highly-studied opening plays, especially by computers, makes the whole process a little too machine-like.
I look at Burning Man for an inspirational example of what a post-capitalist economy could look like; a bunch of people bringing cool art together, not for any monetary gain, but simply because it’s cool to watch people enjoying something you built.
Now, if AI gets better at doing even that (rather than just freeing us from toil by producing radical material abundance) then things could get fraught.
For me the most scary end state (that still involves humans in control and existing) is mainlining / wireheading on generated VR content that is in a feedback loop with brain scanners to maximize hedonic enjoyment. Plausibly this could displace our more-human desires and drives.
But I think there is a big gap between “automation frees us from toil” and “automated maximization of uniquely human art”.
Will AI meet our needs for love, friendship, and self-actualization? That’s more problematic. Though I could still see room for utopia if it does; would the world not be a better place if there was “universal basic friendship” and nobody was completely abandoned? In that world we might still prefer human companions, yet never fall into complete loneliness.
I didn’t intend to claim that BM is actually solving these problems or going in the right direction, just that in a specific time and place and community, there was a glimpse of something very different. (I understand it’s a lot to do with community size and selection effects.)
Side note is that language models are extremely inefficient for the benefit they provide. Most humans don't need to ingest terabytes of domain specific data in order to engage in basic reading comprehension and reasoning.
I'd argue they excel at things like designing molecules, where we don't have the mental capacity or speed to ingest as much information as we would need to reason quickly, but the amount of information and processing necessary to replace a secretary is laughable.
How much data do you think a 6 year old has processed by the time they read a book? Eyes and ears are pretty high bitrate input devices.
Objects and words are tokens. At least, objects and words are not fundamental types of things. Before they are tokens, they exist only through light and sound waves that we perceive (and other sensations such as touch, smell). Our eyes and ears need to process this information before it becomes objects and words.
Seems like a dumb hack, not an advancement indicative of a thorough understanding of the problem.
If and when, then maybe we can use it to bio-engineer our way to a better future?
The idea that children learn from little is naive.
Our best language models at this point appear to be narrow and deep where the child model is wide and shallow. For example, how many pictures of apples do I need in order to train a robot to recognize apples? How many apples (or representations thereof) does a child need to see to identify an apple? I won't argue about image density, but human eyes aren't that high resolution and people don't seem to have excellent high resolution multi-frame memory.
This just makes we wonder if we should be spending more time working on how we utilize our wide but shallow data corpus rather than playing I can't believe it's not statistics with hugely inefficient and focused data sets.
You are doubly wrong because somehow for a robot you are counting pictures of apples, and for a child you are counting distinct apples. The answer to "how many pictures of apples does child need to see" is somewhere in thousands. Remember, brain does an equivalent of 24 fps, so a child holding an apple for 5 seconds gets an equivalent of over 100 pictures.
Too bad they are not saying how long it took to get to the recognition.
Not true. It's a genuine fear of mine that we will create artificial consciousness without knowing we've done so, and doom countless minds to endless suffering before we realize the err.
I think, with regard to AGI, like a lot of A.I. research, implementation will come before explanation.
The idea that computers and intelligence have anything to do with one another, in the manner they are often assumed in the context of AI to have anything to do with one another, is profoundly flawed from the start. A formal system by definition lacks features that are characteristic of intelligence (semantics and the ability to abstract from particulars), and physical computers, strictly speaking, aren't a natural kind nor do they, again strictly speaking, even embody or deal with formal systems per se anyway. Computation is a mathematical construct that is simulated with physical machines, and this model is completely agnostic about the physical means by which it is simulated. You could use wooden gears if you want. The assignment between physical state transformation and a computation is entirely conventional and in the eye of the beholder.
Now, if you want to define "artificial intelligence" as the simulation of intelligence, then I could concede a loose version of this claim, but it would be closer to a marketing claim than a scientific or philosophical one. Practically speaking, such "AI" could still have major consequences as a sophisticated means of automation, but theoretically, there's no intelligence involved except on the part of the artificer.
If you are prepared to worry about your computer becoming conscious, then you should equally be worried about your cup of coffee become conscious as you drink it and subject the contents to the torments of your digestive system. There's no reason to fetishize those artifacts we call computers.
Integrated Information Theory seems like a lot of thought has gone into it, and it would ascribe a good deal of consciousness to some of these systems.
> A formal system by definition lacks features that are characteristic of intelligence (semantics and the ability to abstract from particulars)
And quantum mechanics would have you believe Newtonian mechanics is nonsense, but there are such things as emergent properties, and rules changes as scales allow more complex compositions of simple rules.
> If you are prepared to worry about your computer becoming conscious.
That feels a bit disingenuous. You're dressing down some of the worlds largest supercomputers as "my computer". I'm not worried about my M1 Mac suffering. I'm worried about an emergent mind composed of thousands and thousands of computers working together to simulate intelligence suffering.
I believe very deeply that spiking neural networks are the path to AGI. I also believe that the amount of compute ultimately required to realize practical SNNs is going to turn out to be shockingly small.
If there are no actual time delays modeled within the network, then whatever it evaluates is representative of the outside state at some instant in time.
I think a properly modeled architecture would look something like the actor model represented in hardware, where each actor is basically a small extremely power efficient cpu but unfortunately you wouldn't be able to pull that off the shelf.
This is where we begin to disagree.
If you are content with moving from actual real time to synthetic time, all of the potential frustrations fall away again. Think about online vs offline rendering in computer graphics. Theoretically, you could render all of Toy Story on a RasPi4.
Most of us would be willing to wait days for answers, assuming the batch size is acceptable and the answers are correct.
> Online non-greedy planning technologies that can operate in the presence of a priori unseen tasks, states, and actions.
> A world model technology that robustly predicts state-action-state transitions for all states and actions that might be encountered while performing any task in any environment.
People use greedy strategies all the time, and the intractability of considering 'all states and actions' is the Frame Problem. It is lack of generality that avoids the Frame Problem, and makes planning tractable for animals.
In my view, AGI is an anti-goal. Specialization is what makes intelligent systems feasible. On the other hand, there are technologies like DNNs, gradient descent strategies, even computation itself, that are generally useful for application to specialized intelligent systems.
By analogy, all animals are made of cells, and developed by evolution. But there's no General Animal that can do what all the others can, even if they share core technology.
edit: I wish I hadn't given the analogy. Quibbling about it seems to be irresistible, and has has distracted from my point here.
Humans can artificially travel fast, fly, swim deep, drill deep, move heavy things, kill very efficiently, etc.
It seems like people want it to mean not just general utility but a full simulation of a person (which then sometimes automatically becomes godlike). But they don't say that.
Anyway, decent article. Reminds me of what LeCun has been talking about. https://ai.meta.com/blog/yann-lecun-ai-model-i-jepa/
The big lack remains manipulation in unstructured situations. Although that Google project mentioned a few days ago has some promise. They train on simple manipulation tasks with a force feedback teleoperator, and then use an LLM for the higher level task sequencing. But the simple manipulation tasks remain rather simple. Too simple to, say, assemble flat-pack furniture, let alone repair a car. LLMs just don't seem to be relevant to such non-language tasks.
It's just a matter of time.
The article claims this is insufficient for AGI, but to me it sounds a lot like Kahneman's distinction between fast and slow thinking. Slow thinking is the offline planning, the trained network does the fast thinking. If humans work that way, then a human-level AGI probably can too.
I've sat in countless pitches and meetings with bright-eyed founders touting the imminent rise of AGI. There's a palpable excitement around the topic, but from a purely business perspective, there's an equally strong air of naiveté.
I often wonder if proponents of AGI have truly considered its commercial implications. let's indulge the idea for a moment and say that it becomes a reality. how do you price it? how do you convince industries to adopt and trust something that mirrors human cognition, with all its opacity, when we still grapple with the biases and imperfections of simpler AI models?
add to this the astronomical R&D costs. the sheer capex required is a massive barrier. even with vc backing, the path to profitability seems tenuous at best. and in our current startup landscape and inflation reality, such ventures are money holes.
to me, AGI feels like childish pursuit of the shadows of sci-fi dreams. there's merit in pushing the boundaries of technology, but mistaking theoretical marvels for immediate commercial viability is a classic pitfall. for now, I'll reserve my enthusiasm until someone can convincingly bridge the chasm between agi's promise and its pragmatic business value.
You cannot build an accurate world model without sensory input and the ability to gather data-driven descriptors of the physical world.
1. How much space would it take up? GBs?
2. Would we have the computing power to run "it", say on a top of the line consumer laptop?