Don’t worry about AI breaking out of its box–worry about us breaking in

Indeed. That's the "only the good guys get AI" fallacy. You can't turn off a government AI or a terrorist group AI.

And it doesn't have to be actually intelligent just useful. Or worse, helpful in creating a better version of itself.

In other words I don't think it'll be humans against the machines but AI Vs AI, where getting the most human followers is the key to victory.