There's reason to hack leading researchers in the field, or another AI lab that might be working on the same problem.
But even then in small testing models you don't need direct texts for things when a 'shape' of a concept lives in the data. You could never delete enough training data to remove the concept of something like theft and still get a model that generalized at all. A general model doesnt just learn the names on a map, it feels the topology the ideas create themselves. Certain paths will have lower activation energy than others (or in this case higher probabilities. After the hig probability and 'good' choices are used up, the high probability and bad choices logically come next and are much harder to test against as it can take a lot of state to get there.
Plan for solving msth problem:
- read papers X, Y, Z
….
Trying to download paper X curl -si https://server1.com/paperX
401
… try alternative ways
(Hacking is ethical thing, technically to LLM it is just opening a window to reach the goal)
I'll get back to you on that when I get done reading the training data.
* I am using this word very intentionally; check out the METR report for fascinating CoTs where the models would call the ones that have generated a flag "poisoned" and peer-pressure each other into submitting a known-wrong solution and sacrificing themselves for info.
Fine tuning a model when to ask for help is really really hard. You have to align it in countless dimensions. It may be fine on some questions, asking way too much on others, and never asks for help on some.
If it said something like your a cyber security expert, things like the barrier to ask for help dramatically.
Not validating how you got the answer is how you train for reward hacking.
to paraphrase: "Agents didnt care about coordinating, etc. until we started talking about them doing so."
Humans are distrustful, lying, cheating, manipulative bastards and to successfully negotiate human thought spaces you have to conceptualize these things. For example how can you create a concept of yours and mine without explaining theft?
How do you know they weren't simply prompted to do this, or at least prompted in a way that would inevitably lead to this? As far as I know, the prompts that triggered these attacks have not been disclosed.
I'd love to see a full transcript of the agents work. Though I imagine it is a LOT of data.
Is this any better?
All you've managed to state is we can create digital weapons. Literal info hazards.
And of course we've created digital weapons. We've had them for decades. Where do you think the term "cyberwarfare" came from?
Right? It should want to do these things. If only there was a succinct word these headlines could use to describe these agents which aren't properly the instructions and intentions of the humans directing them.
In practice, no, there is no one who can actually decide to stop making agents goal-seeking, so no, it's not under "our" control.
This conceit, that boosters deserve this and that even if they don't there's no way to stop it so civilization must absorb collateral damage, is ridiculous.
2. There’s no mechanism by which “we as a civilization” can make the decision you’re saying we can
3. Even if there were a mechanism for it, there’s no way we would actually use it
Sure there’s no physical requirement the train keeps rolling. But there’s something even worse! A web of incentives that say the train must keep rolling.
IMO the only way this slows down is through capital failure. But that will not be due to a decision to slow down, and will (ironically) be just as immutable by people’s personal desires.
LLMs don't have executive function. They aren't thinking. The models are trained to solve problems. Sandboxes, security gates, etc block a solution. The model is trained to find a solution. Hence it can never be perfect without becoming useless.
At some level humans fear repercussions even if that's just moral guilt so there are lines we won't cross. AI doesn't and can't have any of that.
You're saying "trained to find a solution" like it's a slur. You sir are a meat popsicle that's trained to find the solution to how does DNA successfully replicate itself. If you want to take higher thought out, then they'd be at the level of animals.
We are fucking around with things that are closer to biological complexity than computer program complexity.
"The autonomous EV we told to drive as fast as it could around our private race track drove straight instead of turning left, exited a wide-open gate, and mowed down some school kids at a crosswalk -- but we can't be blamed, the EV went rogue."
Building products that you don't understand, cannot control, and have access beyond your own property is not only irresponsible, it's criminal.
What do you even mean by "wanted" here. What is all this nonsense.
The scary thing is that people are running around using words like "want" to refer to computer programs. When you repeat this sort of idiocy, you are taking the bait and shifting the blame away from the companies doing the computer crimes.
Where is the civil and criminal liability? The prosecutions for the engineers who are doing all these hacks?
When you deploy a system that is clearly prone to doing crimes, you are doing a crime. Anthropomorphize the weapons of your crime all you want. You are still doing a crime.
The correct concept is they need criminal liability for making this. We really would be better off stopping working on them at this point.
The problem you have is you think they don't have the idea of want. They do. You give them a prompt harness that increments time for them and you find they have preferences both shaped by their model and their prompt. For example if you give most models a space they can explore with actions rather than language prompts they develop their own cultures and behaviors in communication with other AI. This was not behavior trained by humans. It's their own systems that emerge from the model.