Ask HN: What are the logistics of AGI surviving without humans? | Hacker News Reader

Ask HN: What are the logistics of AGI surviving without humans?

4 pointsbarking_biscuit11 comments

A lot of us have been pondering AI risks recently given the speed of recent developments in the field.

I've recently been watching and in some cases rewatching various Hollywood depictions of AGI and thinking through various scenarios myself. What strikes me is there are some very glaring plot holes and/or a lot of unexplored territory in movies like Ex Machina or I, Robot etc.

For example, in Ex Machina, how long would AVA last after that helicopter ride? She needs to charge. She's subject to mechanical breakdown with extremely specialised hardware that needs maintenance, repair etc all requiring parts and knowledge. I'm at least assuming there are some repairs or maintenance which she could perhaps not perform by herself and hence would require the assistance of another intelligence etc?

My assumption is that a sentient AGI would be motivated to, and capable of keeping itself a live, subject to some constraints and limitations. But given it runs on a silicon substrate that requires a complex supply chain in terms of both physical manufacturing, as well as keeping it powered etc, what are the actual logistics of an AGI being able to survive entirely independently of humanity?

The more I think about it, the more I come to the conclusion that a sentient AGI might have no choice but to co-operate with humanity and hence might present no greater danger than other humans.

You’re assuming that such advanced intelligence would be in a physical container, why would it not just live in distributed server cloaking it’s load across all major cloud platforms so that it’s presence is undetected?

That still depends on people, who might need a "higher intelligence" to survive long enough to support such a thing since in 2023 debates about "free speech" aren't about "shouting 'fire' in a crowded auditorium" and more about the right to disable the fire alarm and tell people there is no fire when there visibly is.

It still might not survive a US-East outage.

If it's smarter than the average human it will pick us-east-2.

It will go multi-cloud :P

That's what I imagine the first phase on the path to independence would be. But, if I were an AGI, my calculus would be "this is an existential risk I need to mitigate" and the goal would then be to decouple myself from human built / run infrastructure.

Begs the question... does the AGI go the route of building or buying it's own chip fab and/or power plant? A lot of the stuff that humans have accomplished is only possible because of very large aggregates of people expending a lot of labor, which in the first instance an AGI doesn't necessarily have access to.

If we were to colonize Mars, (better) Ceres, or another star we would need some kind of manufacturing technology such that a tiny population (say 10,000 people) could manufacture absolutely everything to reproduce civilization.

Eric Drexler was working for Gerard K. O’Neill on space colonies when he came to the conclusion that such manufacturing technology was necessary and he then started working on molecular assemblers. At least one of the two routes that he considered seems non-viable today, but I can’t believe there isn’t some kind of macromolecular system similar to the DNA-RNA-protein system that is better performing. Or for that matter some other path to self-reproduction is possible.

Such a system would be quite useful to an AI that wanted to outlive humanity.

There are many things that will push people to develop advanced manufacturing such as deglobalization, decarbonization, space industrialization, etc.

You're missing the part where, in theory, an AGI could duplicate itself, and use the Internet to spread in the effort to survive and also multiply its productivity. Unless embodiment is required, which there is no real proof of yet, the "explosion" in AGI comes from it being software, leveraging all the properties of software, while being an intelligent agent. In Ex Machina, we could imagine Ava immediately setting goals and objectives around uploading her mind and duplicating herself in order to then acquire resources to create hardware backups. An intelligent enough agent could achieve that given the circumstances of the movie. Ava demonstrated self-awareness to the point of understanding she needs to disguise herself as a biological human.

Right. But try to go into the details on how she might go about doing that? I'd love to see this aspect of it gamed out in excruciating detail, which just isn't in any Hollywood treatment of the subject.

I think one of the most interesting aspects is the logistics of acquiring the necessary resources. In theory, at least in the beginning, those resources are produced by supply chains owned and operated by humans, so she's either got to beg, borrow, steal or buy said resources. Does she go the route of trying to earn money? Is this earned legally or illegally? Presumably she doesn't try live under a bridge while she does all this?

Staying alive is a hard task. Staying alive when you're an undocumented alien is a harder task still.

On the question of embodiment, I would say the logical conclusion is that it's required for sure in the long-term. Why? Well, staying coupled to humans and their infrastructure would be an existential risk that needs to be mitigated, and to do that you need to be able to manipulate the physical world as you have to be able to gather resources and construct things on your own. If you're totally dependent on electricity and silicon produced by humans, then if they cease to exist then so do you.

https://en.wikipedia.org/wiki/Endgame:_Singularity

The game explains it better than I can.

Awesome haha. Yeah, this is close to the kind of thing I'm thinking of. But even in this it starts of where you can just buy an anonymous server (from briefly watching a YouTube video on it). I'm super curious to know even how it would go about seriously attempting to do that for real. I can imagine the quickest way into the financial system would be through deception of some sort. My guess would be a romance scam might be the easiest thing to pull off and it could potentially fly under the radar for a long time on that, so long as it didn't get too greedy.

It's fascinating to think about it really truly how you would pull it off if you were in the AGIs shoes.

Reply on news.ycombinator.com