Indeed, and it's just going to end up as a zero-sum game where the winner is the one who can most adeptly manipulate the machine to cause harm to the other.
Of course we should proceed with even more powerful tools. We also need even stronger governance of said tools. We don't want an AI-version of the Chernobyl incident but we do want the AI-version of the benefits that nuclear power gives.