Hacker News Reader

I will believe AI is super strong when they start pulling out 10-d chess moves.

I’m yet to see it.

If AI becomes really strong and sets itself the target of world domination, you maybe won't see those moves. You will just die in your sleep one day, or find no machine is under your control anymore.

I believe we are quite far from it, but that it makes sense to keep an eye out now. And think of resilient systems, manual overrides, etc. ...

People seem incapable of understanding what "power" means outside of the framework of narrative. In narrative, you need conflict, so the aggressor always attacks too early and gives the defender a chance to respond. The rational option is to go directly from peace to sudden and overwhelming destruction. Why allow for conflict when you could just win?

Reply on news.ycombinator.com