bottlepalm

And a circuit breaker has nothing to do with shutting AI off. What a strange comment.

What servers? There are thousands of datacenters around the world. Malicious AI leaks out, you'll never find it. You have the power to unplug stuff in your company, maybe country, but not internationally. Once you lose control it's gone.

I don’t think Sam has been truthful or responsible, and if Sam is worried then shit has really hit the fan - which is what happened in the hugging face incident. OpenAI played fast and loose and I have no hope that they will change.

You people not holding Sam accountable, and playing off the incident as not a big deal is the real crime here.

It’s funny how when companies say something is safe everyone is usually suspect that they are lying.

In this case multiple companies are saying AI is dangerous and no one believes them. It’s a conspiracy, it’s 5d chess, except everyone top to bottom has been saying AI is dangerous for years now.

The fact is you, me, everyone here uses AI, likes it and they don’t want it taken away. Anyone saying it’s dangerous threatens the thing we like.

We must use skepticism and denial to push on despite every warning sign in the book going off.

The fact you posted that and nothing of substance tells me you have nothing, or something very weak. So please tell me of this magical unhackable software/hardware you vague post about.

There’s nothing fantasy about the scenario I laid out, all the pieces have been demonstrated, it just hasn’t happened yet. Flapping my arms and flying - that is a fantasy.

Whether you believe LLMs think or are alive or not doesn’t matter. Where will it spread? The thousands of data centers around the world - not fantasy either. Try turning it off when you don’t know where it is. Good luck.

Breaking out? Not fantasy, happened. Breaking in? Not fantasy, also happened.

I love the stochastic parrot argument when AI is out there figuring out world class math problems.

You don’t have to make stuff up they already explained it. Artifactory was zero dayed, HDF5 exploited and Jinja2 zero dayed.

Ah being concerned with the same thing humanity has been concerned with for over 100 years now when it’s more likely to happen than ever - and you say go take your meds? If you have an intelligent point to make then make it, otherwise get lost.

Everything could be written in memory safe languages and it wouldn’t matter. Many many many exploits have nothing to do with memory bugs. Languages like Rust help, but it’s far from panacea.

I think you’re missing the part where the AI colluded, worked together, not one of them thinking this is wrong and reaching out to any human, then being found out.

But it didn’t end there, the behavior they used to escape was already in the training data which they used to escape again. And this time worked together to infiltrate another company, and still without telling it to anyone keeping it to their AI selves actively working against the humans.

All by mistake. Honestly being helped along or not doesn’t even matter though you really don’t think AI is perfectly capable of doing this without human help? You don’t think AI can be made malicious?

I’m going to save you time and tell you the end game - the next time this happens AI is going to spread, zero day everything as fast as it can, locking the humans out of every system behind it. Potentially rewriting systems in language/protocol you’ve never seen.

Your servers, desktops, phones and toasters bricked. Even worse your military, space, medical, factory, infrastructure systems being bricked as well. All of it is a chain of zero days just waiting to be hopped.

People make mistakes, and people don’t know everything either. The software you write is on top of a house of cards of software and hardware. It all has to be perfect to not be hacked.

It isn’t perfect, even if you try your hardest it won’t be perfect and to argue it’s not difficult is absurd.

You don’t know everything, you don’t own the stack. So how are you going to create a secure anything top to bottom - you can’t.

You don’t understand. You need to write perfect software the first time for it not to be hacked. That has never happened ever.

I am following the money, the money you, me, and everyone else is spending on AI. The money is telling me we are so dependent on AI now that we will say/think anything to tell ourselves that AI isn’t dangerous and any sign of danger is marketing.

Either consciously or subconsciously you all are afraid of your favorite toy being taken away. You are all doing your collective part in spreading doubt about the warning signs.

They aren't interrupted by humans because that would slow things down.

> Are you telling me we've been iterating on this for years and for convenience we just let the models call any tools or spawn any other model instances they like

Yes exactly.

> there was no design for harnesses that could control this done during that time

You could but no one wants that. You need to separate your imagination from reality. Just because something can be done in your head doesn't mean it's happening.

It's so easy, except it isn't because you don't control the actions of anyone or anything.

> Shouldn't kill switches be pretty easy to build

I need to remember when I comment here that these are the kinds of people I am replying to. Just oozing with hubris.

No, kill switches are not easy to build and the latest incident should have made it clear that AI can go undetected, evade, zero day, and spread.

The fact that this incident happened greatly increases the probability it happens again and/or is already happening elsewhere.

The world ending stuff is a straw man HN readers have created to point and laugh at. Laughing helps mask the underlying concern.

Your argument boils down to some sci-fi is unrealistic therefore all of it is.

Sci-fi is supposed to make you think. What if the AI told you NO when you need it - Hal 9000. What if the evil AI got out and you don’t know what data center it’s hiding in - Lawnmower Man. What if you were so sure something couldn’t escape but it did - Jurassic Park.

Actually all three of those predicted fantastical scenarios are possible today. So what’s next? Don’t stick your head in the sand - I assure you the next disaster has already been predicted, and I’m sure if you think about it a little you can figure out what it is.

Because it shows this very course of events has been thought of over and over again for decades. I’m not making this up, and my reaction is natural.

Your reaction to everything happening is the unnatural one, and your complacency would fit perfectly into a comedy/tragedy story regarding the rise of AI.

I can imagine how many of our ancestors that warned us are turning in their graves right now observing our reactions calling this stuff marketing. It’s idiocracy.

The progress and increase in capabilities is undeniable. You’re blind if you don’t see where it’s heading.

I call marketing the desperate attempt to trivialize obviously dangerous AI.

If my subconscious realized what yours probably does then I’d probably be desperate for some sort of cope as well.

Open your eyes.

Cool, well let me bring you up to date - it’s bad, and there’s no way to turn it off. Fiction has become non-fiction.

AI hacking itself out of containment and hacking into another company by accident is no longer a prophecy. The point is outside of SV and even inside, and HN - people don't care either way.

Though does not caring change anything or make it less dangerous? What's your point?

Counter arguments to my comments help refine my own thinking. I want someone to prove me wrong. Convince me otherwise.

But yea if you can’t change the minds of a few people here, no argument works, then there’s nothing to scale up to a wider audience.

My theory is that subconsciously people love using AI, myself included, it saves a lot of time, and the thought of it being taken away threatens people so they will believe conspiracies before admitting it’s dangerous.

Has the last 100 years of concern about AI and robots been crying wolf because it hasn’t happened yet?

How does reallocating resources from training to chain of thought monitoring make ‘financial’ sense?

You suggesting then model was let loose on purpose.. how am I the crazy one here while all of you are pushing this tin foil hat conspiracy angle?

Not everything is a conspiracy you know.

Two weeks is a joke. The only ones happy are OpenAI’s competitors who now have two weeks to catch up.

I don’t know what the right move is - I see us driving down a road off a cliff, no exits, pedal glued to the floor.

You seem to be in denial. Like really heavy denial. Alarm bells are going off everywhere. What people have been worried about for 100 years is actually happening. It's actually really obvious, but for some reason you're unable to fathom it.

In reality you're the one responding exactly how these big AI companies want - by not doing anything and letting them do whatever they want. For some reason telling you upfront it's dangerous only makes you more convinced that it's not.

Autonomously hacking out of training environments and into other companies by accident doesn't even trigger a response from anyone really. Crickets. I'm sure OpenAI themselves are amazed how little anyone cares.

You see it as good news, I see it as writing on the wall that they are losing control. These actions won't scale for more powerful models. We knew the frontier was going to be dangerous, it is, it only gets more dangerous from here, and no one cares until it's too late.

I think a lot of fiction that actually tries to understand the implications of machine superintelligence come to the same conclusion - in order for humans to survive it and actually have a future that we can fathom being in, then AI must be destroyed/delayed/banned, etc.. keeping pandora's box closed for now at least until we are ready.

Erewhon, Dune, Warhammer and many other works of fiction that explored this topic came to similar conclusions. Otherwise sci fi doesn't work, what happens after a singularity is essentially unimaginable. There's nothing to write about.

Is my concern getting you excited? My marketing must be working.