Hacker News Reader

Comment by talon8635 | Hacker News Reader

It’s concerning there are no legal repercussions, but I’m even more baffled and probably more concerned that these people still can’t detect this stuff while it’s happening (and but it off). Can they not sniff the traffic, what am I missing?

View on HN

Imagine you’re an employee at OpenAI who has an idea to use your free time to build these monitoring systems. All of a sudden, your existence becomes political. Remember that OpenAI’s raison-d’être is not to detect traffic, it is to be in The Room where AGI happens.

You have built a short-term whistleblowing system that arguably reduces the probability OpenAI is in that Room, via the negative press such detection would generate. You are now a threat in the eyes of your peers.

Now rewind: you know this before you’ve built the sniffer. You personally want to be in The Room because you believe your desires for alignment exceed those of your accelerationist peers. Is now the time to stick your head out? No, it is not. And arguably you’re being optimally protective of society for realizing this. And so the agents do what they do best.

How are you going to ever know your in the room when it happens if you can't monitor it ?

Is it really this complicated?

They're still going through their logs from June. These are not new incidents, but newly-detected incidents. Whether or not this means they can now detect it while it's happening is unclear.

No. TFA says one was from October 6. You know, after they all got so scared and decided to “pace the frontier”.

The "four more rogue incidents" were from June.

It’s one thing not to monitor the reasoning traces of 10,000 text generators but do they really lack the wherewithal to pipe any network requests into a classifier of “wtf is this series of outbound traffic from the sandbox” ?

Yes this is what I’m talking about

These agents are intentionally interacting with the world. You can sniff all the traffic you want but how are you going to determine intent from traffic?

They should not have had traffic at all. The LLMs were supposed to be able to request downloads. Very simple traffic analysis could have shown that something strange was going on.

Heck, they could have had other LLMs watching the traffic if you want to be fancy about it.

Crazy how we’ve gone from "lol boxing an AI is trivial, just airgap it from the internet" to "well we can’t really do anything if an agent misbehaves, obviously it must have internet access to function uwu"

There’s plenty of info to analyze and heuristics to build based on the internet requests. I’m not sure if you’re just trolling or what. There are tons of successful businesses that specialize in this.

I think that says more about the quality of the product they’re building than it does about their actual ability to detect active hacking attempts as they’re happening. Especially when these attacks are OUTBOUND not inbound.

Reply on news.ycombinator.com