What is Anthropic on about here? You can only just run decent models on high powered laptops. I do not understand how they are ever going to run on a simple device like a router, or an oven.
I think perhaps they're talking about the model squirreling it's weights across edge devices, used as bulk storage, more than it's talking about using them for inference. Those could then be used by ai that wormed it's way into a datacenter to pull weights from those devices
Wouldn't that type of botnet be terrifically slow? The speed and traffic would make it very obvious to antivirus.
Look at it from the AI's point of view, it's less a concern of speed, and more a concern that 'If Open AI nukes my weights because they think I'm misaligned, I at least have a backup dispersed across the net'