I guess I'm coming at it from the opposite direction. Lets assume the some entity creates AGI. Which entity would you rather it be?
1. Government - it'll prolly be hidden from the public under the garb of national security, and used for god knows what
2. For-profit companies - Google, Meta, MS, etc who have shown themselves to be untrustworthy and purely motivated by profits
3. Non-profit companies - OpenAI (and others?) who have explicitly made the commitment to optimize for long-term benefit, safety, and cooperation [1]
I'm not saying there is a right answer, but the non-profit route seems like the least worst option.
> Non-profit or not, AGIs shouldn’t be controlled by anyone.
I don't see that ever playing out in the real world. It is much too powerful a tool to be made accessible to everyone.