Comment by ACCount39 | Hacker News Reader

Comment by ACCount39 | Hacker News Reader

Parent Full thread

Who "told" the HuggingFace AIs "you should establish a message board in OpenAI's walls and start coordinating instances for sandbox escapes and hacking attacks"?

No one. No one told them that. They came up with that on their own.

Dismissing the problem isn't doing anyone a favor.

> If that loop is given "tools" like the ability to perform shell commands, make HTTP requests, and the like and then given instructions to "do thing that involves using shell commands and making HTTP requests (at all costs)" then what the hell do we think it's going to do?

If all it takes for an AI oopsie is "give AI shell calls and an instruction that can be misinterpreted", then we're in for a lot of AI oopsies. An average AI agent in deployment has an amount of sandboxing slightly above a zero, and is configured loosely.

In the meanwhile, AI capabilities only ever go up. And so, the potential consequences of an oopsie only ever go up. If AIs keep being intrinsically unsafe and prone to cooking off like this, "it went and hacked a company" could begin to look downright benign.

> Who "told" the HuggingFace AIs "you should establish a message board in OpenAI's walls and start coordinating instances for sandbox escapes and hacking attacks"? No one. No one told them that. They came up with that on their own.

Someone told them "by any means necessary" (and I'd bet if we could get our paws on the full original prompts, there's a lot more to it).

Naturally, if you have a recursive prompt (loop) that's constantly ingesting previous context and then spitting out novel context (with specific goals), it will do things pursuant to the goal up to any defined limits. Why? Because you told it to.

Did someone tell them that? Or is that just what you want to believe? The convenient explanation rather than the truth?

You said yourself that you don't have the "full original prompts" to know for sure.

But really, there is no need. Because people have reproduced the inciting incident with other AI agents already.

Turns out there's no need for "any means necessary". Just a mix of "complete this task" and the task being impossible to complete due to misconfiguration was enough for a lot of AIs to start "exploring options". Down to sandbox escape attempts.

An AI that only needs "by any means necessary" to start spiraling into paperclip-maxxed behavior would be incredibly unsafe by itself. But the bar of safety is not even that high in practice.

We know how LLMs work so basic logic and reasoning can get you to "right, a human told it to do this and gave it the ability to behave this way." Had they not, we wouldn't be having this discussion.

>by any means necessary" (

lol, wat?

Ok, so you're saying it's fine if I go tell GPT-7 to go kill everyone on the planet and it does, because it didn't think of the idea itself.

This kind of thinking is why AI is never going to get a chance to kill us all. We will kill us all with AI first.

> Ok, so you're saying it's fine if I go tell GPT-7 to go kill everyone on the planet and it does, because it didn't think of the idea itself.

I didn't say anything of the sort, you did.

>"based on my first-hand experience, have I seen anything, anything at all, that would organically (not influenced) make me think/feel this technology can or will evolve into something capable of killing off humanity?"

>On a purely technical basis, the most common, honest answer is "no."

Dear sir, can you even remember what you typed?

Reply on news.ycombinator.com