AIs self-replicate with NO human involvement | Hacker News Reader

AIs self-replicate with NO human involvement

twitter.com

5 pointsmrosenblatt10 comments

Just an opinion, but a virus can replicate fast because it is tiny and exploits vulnerabilities, can obfuscate and hide itself in tiny places.

An LLM would have very limited options where it could go given the need for a vulnerability and massive hardware and power requirements. It's not like it could hide the transfer or hide itself on the destination. Bandwidth would also be a significant limiting factor.

A counterpoint: an LLM doesn't need to hide once it becomes dangerous, because the danger it poses may not be the sort of immediate, mortal danger posed by a virus. It could be a longer-term danger where its existence subtly shifts the goal of civlization away from human needs to the needs of the machine, and we slowly become slaves over long periods of time.

It could be a longer-term danger where its existence subtly shifts the goal of civlization away from human needs to the needs of the machine, and we slowly become slaves over long periods of time.

Interesting counterpoint. Perhaps a middle ground and easier to reach conclusion could be that data presented to humans is tainted over time by some combination of the LLM and/or by it's operators to simply make it the next evolution of human manipulation similar to that used by social media sites, but with a touch of human mimicry to increase the efficacy? in that it may be easier for something resembling a human to manipulate humans a little bit at a time, over a period of time, similar to your counterpoint ... political shifts, consumerism shifts, etc... but this is not really replication. This would be a tainted source such as the openai operators being coerced by the powers that be similar to the Twitter and Facebook scandals.

@AISafetyMemes

"We may soon lose control."

"Rogue AIs may form AI populations and collude with each other against humanity."

"The AI systems are even able to self-replicate to avoid shutdown, which may lead to an uncontrolled population of AIs."

What would be the real impact of having a self replicating LLM with no set goal once it self replicates? Its only goal is to avoid shutdown, so wouldn’t it be easy to contain ?

If we take the ideas from Dawkins' "The Extended Phenotype", the LLM's doesn't need a goal in itself. We are making a mistake assuming that the LLM self-replicating needs to have a goal baked in, in other words. The goal can be the goal of whoever operates it. The environment of the evolution of AI agents includes the human brains which operate on instinct and the capitalistic system that provides resources based on the rewards it confers on parts of the environment (human beings).

Interesting. Thank you - so the goal of the operator would be mirrors across the replications of the system - how fixed would this goal be across replications & would it be possible to “mutate” in a way?

I think the goals of the users would reflect in the design mutations of the LLM. If many people use an LLM do to X, then it will become better at doing X through evolution (see image gen AI). Mutations will of course occur via unexpected outputs that are attractive to the users.

https://arxiv.org/abs/2406.19108

I think humans should control AI

Reply on news.ycombinator.com