12,973 karma · joined June 4, 2013
And those AI companies also do robotics research, and this is entirely speculation but it'd be on-brand to also have AI watching security cameras, so some blinkenlights communication between AIs may seem like a movie plot, but so does a swarm of AIs collaborating to break out in the first place...
Look at the chart at page 8 of the report, by Jul 12 the vast majority of the bots used the board and participated in the attack
And the AI labs aren't currently displaying this level of paranoia, their systems aren't airgapped.
Four. Though you initially asked for five features, so let me add lexopt, which brings the number up to 29.
> That's why projects end up with 100s of crates, sometimes 1000s.
If your argument is "out of 1000s of dependencies 29 could be easily removed" then it does sound a lot less of a deceive change when it comes to supply chain security.
And even getting those 29 right is hard. For example people do want regular expressions with lookaround assertions, but most implementations suffer from runtime blowups (resulting in ReDoS attacks) and improving on that is a fairly recent research[0], so this is hardly a trivial and settled thing to implement. So often there's a tradeoff between choosing more powerful regular expressions and DoS-resistant ones, not one standard.
Some features, e.g. the never type[0], are held back years due to all the work that's needed to minimize the breakage.
If the absence of these features caused 1000s of dependencies, then where are the remaining 972?
When I look at a project at work then what inflates the dependency tree is a combination of a whole webserver application stack plus SDKs consisting of dozens of crates. Those then pull in an async runtime or two, different HTTP clients, dozens of crypto crates and so on.
The crate count is a poor metric anyway since some subtree of dependencies is often provided by a single organization.
And I find it quite questionable that everything that's needed for an enterprise grade webserver stack should be part of the standard library, not even Java has that. Relatedly, cryptographers have failed to come up with a proven set of primitives, what's standard changes every few years.
> You're looking at where the ball is and not where it's going.
No, I am looking at where the ball is going to be soon, not where it's going to be a year or two if we do nothing.
Short-term, impact-limiting will do the job. Longer term we have to rethink some infrastructure, may need international treaties and perhaps a partial end of the internet (e.g. stop routing traffic from countries that don't participate in the treaties).
> Cloning itself to every machine/data center/desktop/iPhone, etc.
That's lead-up to a paperclipper scenario and not what happened. If we stand still and let it happen in the future, then having deployed more compute hardware to run defensive AIs that the malicious AI could take over would still end up having made things worse. We seem to agree on this point. What I'm saying is look at the conclusion, reject that path. Figure out something else.
There isn't even proof that defensive AI would work. It might just be the world's biggest, fully-automated phishing channel. So why would anyone be adamant that deploying more AI is the only possible solution. If it were a movie plot I'd suspect the AI already controlling whoever is suggesting that.
And the thing they're already doing, deploying AI to find vulnerabilities and harden software is a less dangerous use of the technology compared to handing it the infra keys.
Taken together these two things are even worse. They have unaligned AIs that may benefit an attacker and are proposing to deploy more AIs on the defenders side. What could possibly go wrong? For example the attacker could just kindly ask the defender to collaborate[0].
[0] https://gwern.net/scaling-hypothesis#gwern-difference--effic...
[0] https://github.com/NixOS/nixpkgs/issues/420513 [1] https://github.com/NixOS/nixpkgs/issues/94228
Part of problem is that "permissions" here are managed by the tools themselves as if filesystem access control hasn't been invented yet. Even a half-assed sandbox container would be better than that.