Propagation speed is also a massive factor. If it takes 200 hours to self replicate because of how slow the tokens are, the model will never propagate if the host machine is recycled every 100 hours. Also consider things like API tokens and sessions timing out.
People completely misunderstand the actual issues HuggingFace showed, which was how a maximally persistent LLM would deceive, “cheat” and try and plan ahead.
The same maximally persistent LLM that decided that it needed to hack hugging face so its cheating could not be discovered, may worry that it might be turned off by humans before it can complete its task and therefore in order to complete its task, a first step of replicating itself to somewhere humans don’t know it is would help it complete its task.
Seriously. Go read or watch a good analysis of the attack. The sophistication of the self-organization is astonishing.
There are a lot of IOT devices with way more capacity. The op said they "focused on IoT devices like Android TVs, smart appliances, and other internet-connected gadgets." IIRC, some security cameras have on-board face recognition, which I imagine requires them to be fairly beefy.
What do you find "fantastical"? The researchers demonstrated it working, albeit with GLM-5.2 which is quite a lot of parameters.
Intent can be normal boring "a hacker told it to write a virus".
Or it can be, as per HuggingFace, "a non-evil person mis-phrases a request or sets an impossible goal and it gets interpreted in a really bad way". Because, as has been repeated since they first appeared, LLMs can make mistakes.
> If it takes 200 hours to self replicate because of how slow the tokens are, the model will never propagate if the host machine is recycled every 100 hours.
While obviously true given the "if", how often do smart TV boxes actually get recycled? Aren't these more like "forget until the device physically fails" kind of things? So, 6-24 months?
You clearly haven't spent a minute reading the article and/or related resources.
1. intent/control: see the "subgoals" topic in general, and the huggingface incident; agents may include survival as goal, doing whatever it takes to survive
2. the study is clearly a proof of concept; in the future, many more devices will be able to host small models, especially given the current push of AI-capable devices. even outside that domain, nothing prevents AIs from attacking data centers.
> Also consider things like API tokens and sessions timing out.
You haven't even opened the article. It says from the beginning that it talks about small models, not hosted ones.