"Lock the weights, weights are dangerous" again. Sorry, keeping such power in few hands is inherently more dangerous than distributing it.
AIs do not kill people, people kill people.
AIs do not kill people, people kill people.
A simple thought experiment - imagine AI discovers a way to make nuclear bombs capable of destroying the entire world radically more cheaply and accessibly with common items anyone can purchase. AI can help anyone build these nukes trivially. Even if the AI and the nukes don’t kill people on their own, do you want literally every human on earth to have this power in their pocket? Do you think it would be a net positive outcome for us vs a world where that’s not the case?