Yet nobody has a clear practical and reasonable scenario where an AI will cause human extinction. Pls don’t say AI will control the launch codes and start firing randomly.
To give one example, an AI could be instructed to reduce the crime rate, but rather than preventing people from committing crimes, the AI could decide it’s easier to prevent all victims from reporting crimes to anybody. That would remove the threat of law enforcement that currently prevents many people from turning to crime. Such a society probably wouldn’t cause human extinction, but would still be horrific and worth preventing.
If you are skeptical that AI that is trapped in software could gain any significant real-world power (not necessarily the power to kill all of humanity), https://slatestarcodex.com/2015/04/07/no-physical-substrate-... describes some possible mechanisms.