Hacker News Reader

The opposite of "open" isn't necessarily "closed". It could be "classified".

There was classified public key encryption before there was open public key encryption.

The difference is that classified public key encryption was developed by a public, albeit secretive, government agency. The risk that the OP is worried about is that a private entity would control a powerful AI. If AGI is as powerful as people say it is then it would give a private company an unprecedented amount of power, that is perhaps a greater concern than the dangers of AI itself.

A government agency would have greater means, motive and opportunity to misuse AGI than would a private entity, which is more subject to countervailing influences, including the government. So I would think that a classified government AGI is more dangerous than a proprietary AGI.

But in any event, I think that AGI dangers will depend strongly on how AGIs are connected to systems to gather information and how they are connected to systems to take real world actions on their own.

Reply on news.ycombinator.com