Great, so as long as alignment guardrails always work and we don't accidentally tell them to solve an impossible problem, humanity is safe?
Sorry, but that's completely delusional.
Which misses the point entirely because the hack itself was inconsequential. No data was actually stolen, and it doesn't seem like hugging face actually suffered any losses aside from whatever effort they need to expend to investigate / clean up the hack. Neither the victim or the government wants to bring legal action. What was consequential is proving that AI can autonomously hack systems and do it unintentionally.