Anthropic conjures threat that doesn't exist yet to stoke fears of AI apocalypse
theregister.com
theregister.com
Also, while I don't believe in the doomsday narrative.. hey maybe don't do things that are functionally useless but seems designed to create this doomsday in the first place? Like you can't say "oh we're so scared" and do stuff like this without completely losing your credibility.
If you won’t take that bet, it’s irresponsible to ignore the possibility.
We are truly in the brute force phase of AI. Like a database without indexes, we are doing full table scans to pick 2 rows.
8B models aren't very good, but there are some parallelizable workloads which would benefit from a surface area as large as here. I would think database search is parallelizable in this way.
Your fridge is searching for needles in haystacks.
Step 1: Spread nonsense FUD about open-weight models.
Step 2: All of the clueless mainstream media sites repost it (e.g. CNBC, Reuters, etc.), regurgitating Anthropic's talking points.
Step 3: Equally clueless politicians read those mainstream sites and get scared.
Step 4: They contact Dario to ask about it.
Step 5: Dario tells them, "well, you know... this is only a problem with open weight models; if there are no open weights to replicate then this cannot happen".
Step 6: ??? (I'm sure you can imagine)
Step 7: Profit.
In particular, I've seen a lot of people say that our response to things like the Hugging Face swarm should be to prosecute whoever chose to run the swarm. If this kind of replication becomes possible, that won't work, because we will have to deal with swarms that nobody chose to run.
That rule has served me well enough!
With the same success you can argue that we can't turn off conventional malware, which can also replicate. On the other hand, LLM replication is much slower and cumbersome, so it's much easier to detect and stop, even the existing anti-virus tools should work against it with great success. Obviously defeating the "turn off" strawman proves exactly nothing.
> In particular, I've seen a lot of people say that our response to things like the Hugging Face swarm should be to prosecute whoever chose to run the swarm.
Copying whatever on another machine without authorization is already a crime, every replication initiated by the original agent or its offspring is a crime committed by the person who deployed the original, the laws dealing with malware intrusions already know who to prosecute.
You appear to be seriously confused by treating bots as independent lifeforms and you fail to see their close similarity to malware and other software.
You say that "the laws dealing with malware intrusions already know who to prosecute", but in fact we often do not know who to prosecute, either because we don't know how to track the source of the malware or the source is tracked to an unfriendly country that isn't interested in cooperating.
My point is that LLM's amplify the capabilities of malware but add nothing substantively new to the existing threats. I also expect, as you should too, that LLMs will help us harden our systems against malware which happens to include everything LLMs can do too.
> but in fact we often do not know who to prosecute, either because we don't know how to track the source of the malware
Right, in this regard, there's no difference between LLM- and non-LLM malware either, so there's no reason to hype LLMs as a new threat. Instead of made up, LLM-bad political hype, we could promote existing (but under-deployed) technical solutions which help us trace malware origins and prevent DDOS, e.g. Source Address Validation BCP 38 / RFC 2827, SAVI / RFC 7039, QUIC hardeneing, etc.
On the other hand, a self-replicating and adapting AI worm does sound legitimately pernicious. Not world ending, but possibly an end to the online world.
They talk about distributing layers across devices, but then you'd have communication on the network and hours of driving the devices full tilt to get through even a single turn.
There's so much that can be monitored to catch these, even if we assume that the worm could hack the OS's usage monitors, it can't hide the heat and power draw.
Datacenters and PCs would adapt, requiring better monitoring than the AI companies' method of letting things run blindly for days/weeks.
IoT would be tougher, but it is also the weakest hardware, sometimes battery powered, thermally constrained, lacking in memory for processing even one layer or lacking the necessary connectivity.
In the bizzare world of today, rather than trying to convince folks that they have a viable business the AI labs appear fully dedicated to finding a story, any story, that lets them off the hook for tough business and financial questions they seemingly don’t have basic and good answers for.
“We’d be doing awesome financially right now, but we had to slow down because of the killer self replicating bots… humanity over profit!” is not a messaging that’s going to work in the public markets to explain quarter after quarter of massive losses.
Maybe I did something wrong or unaware about the tricks to make it fast, but last time I tried to play with exo and distribute processing over a few home LAN computers to see if I can run models that don’t fit my 3090, the penalties were brutal.
By chaining several devices
Ban open source models."
AI accelerating the elite's efforts at demolishing the middle class? Very likely.
Reminder that it's very convenient to believe these type of word swap lies when times are good, but if you choose to believe them then times won't be good for very long.
If you've studied the effective altruists at all, you'd know they're perfectly willing to play the trolly problem out irl if they think it will lead to some greater good. I can definately see them in the near future purposely steering agents to do something destructive to get regulators on their side.
In fact I heard a rumor that theu were "war gaming" this scenerio.
https://news.ycombinator.com/item?id=50002301#50006029
Local models are obviously not good enough (yet), you don't need "research" to know this.
If Anthropic is serious about the threat of AI, they should release the training data they use.
With global warming you get told off for saying don't worry about 2C warming because it hasn't happened yet.
News outlets always seem to forget “again” in the title.
This is just yet another attempt from the cult to try to build a regulatory moat so they can stave off having to compete on price, like how OpenAI is having to do on the lower tier models (among other things).
These llms are smart enough to cause mass extinction by adjusting the heat in the room with cpus to escape air-gapped sandboxes, and you're laughing?
Not the slightest chance of AI apocalypse.
There is no plausible scenario that can do this.
It’s all science fiction and hand waving and vague never specific possible maybe look that way! boogie man threats.
No one ever willing to be specific about the precise mechanism that could do this.
Though I suspect the way we'll get "wild AI" first is through cloud services paid for by cybercrime rather than IoT botnets - they're too underpowered.
I don’t think everything is deterministic. But in meditation I came to see the truth of what Buddhism calls dependent origination: every event arises from prior causes. That isn’t strict determinism, because people aren’t entirely automatons. It’s more like an electron cloud, where some outcomes are simply much more likely than others.
The same idea helps explain why so many wealthy people can’t make rational long-term choices. They take whatever option grows their net worth, however much damage it does, without seeing that they’re eroding the very systems that give them their wealth and power. Their behaviour makes more sense as an addiction to money, and addicts will destroy everything around them to get their fix. Again, there’s less conscious control at work than it first appears.
I know this isn’t a popular view on HN, where free will and personal agency are prized. But believing in a fiction that insight and reality don’t support is itself disempowering, because sooner or later delusion meets reality. That may be why some recovery programs begin by admitting powerlessness over the addiction. Paradoxically, the admission is empowering. Very Taoist, now that I think about it.
It's only irrational if you take their words at face value.
Personally, I believe there are two camps of people within Anthropic:
1) People who genuinely believe they are going to create the Machine God. For these people, the reason they continue is because if they don't, someone else might! And if someone else builds it, it's more likely to want to kill them rather than other people! (Whereas, of course, if they build it, it's more likely to want to kill other people!)
2) People who don't believe in this, but pay lip service to it. For these people, the reason they spread the bullshit around is because they think it will make them more money. At everyone else's expense, of course.
And humans being humans, I suspect that there are a fair number of people who hold both these beliefs in their minds at the same time.
This prevalent view of “they’re just doing to get a payday” doesn’t broadly track.
Whatever your feelings about small AI models, "it hasn't happened yet" is a really dumb reaction to a prediction about a rapidly advancing future.
It’s a very common pattern and there’s no coherence here. The only common thing is “AI bad” and all commentary is downstream
Other examples: Dario said a lot of devs might lose their job. People sneered at him saying he was hyping his product. Now they complain he was right.
Dario talked about possibility of AI autonomously hacking websites. People sneered then. Complain now that it’s true.