> Then the disturbed human tells the refusal-free model to do anything necessary to launch nuclear weapons.

If a model on rented hardware could manage to launch some country's nuclear weapons, the same could be done by model-armed or model-free humans, spies, etc from other countries - using their specialized tooling they could always do better than any model.

In short, AI doesn't add any new threats to the mix, the defenses against AI are the same as against adversarial humans, and the whole brouhaha is nonsense. Sure, make your defenses impenetrable, use AI for that and it'll stop any I, not only AI.