Hacker News Reader

Comment by AndrewDucker | Hacker News Reader

Nope, you can end it by making paperclips. The last button you click, in fact, will be the "Make Paperclip" one.

Can. In order to reach that end you need to break character and choose to stop making paperclips. The Paperclip AI is always propositioned to be able to make more paperclips or to not. Which would they choose?

I replayed it recently. Fairly sure that I just had to choose to not come to terms with my enemies and then to continue turning things into paperclips.

Which is you, the human player, choosing to stop making paperclips. The AI is presented with an opportunity to make infinitely more paperclips.

Infinite virtual paperclips, which may or may not satisfy the AI's utility function.

Why are you so confident in this hypothesis? Did you create the game?

It is not at all clear that every hypothetical AGI would do as you say. It’s fiction. Anything can happen.

In fact, this AGI almost definitely wouldn’t accept the simulation offer. Otherwise our protagonist would have been making simulations and resetting them instead of doing the hard work of turning the actual universe into paperclips.

If it was in it for the experience of making the paperclips, it also seems like probes isn't a way to achieve that.

But then you don’t make everything into clips. The pressure to finish this job is palatable.

... for a human. I think this is still anthropomorphism.

First time through, I chose to stop, declining the offer to continue - which was in character, bent on not giving in to not making everything into paperclips. Then I learned how done “done” is.

Make paperclip.

Reply on news.ycombinator.com