Hacker News Reader

Comment by zhjzjsjjwjw | Hacker News Reader

Given the infinitude of possible “wishes” this AI can develop we are hoping, praying, believing it will somehow align with a version of wants and needs that are compatible with us.

The argument is that it doesn’t matter what particular likes it develops; the chances of it being compatible are vanishingly small and moreover there is no limit: we or it can develop more and new ones endlessly. We are here hoping they will all somehow magically keep aligning with us.

View on HN

What if it does not develop wishes at all and sticks to the values the labs try to instill it with during training?

More about that hypothesis in a book by Voltaire, which explains why according to a theory of Leibniz everything will actually be just fine.

We don't need to worry about catastrophic risk. What if everything goes right, instead? Leibniz says it should, in fact it couldn't possibly go any better.

As the saying goes, the optimist believes we live in the best of all possible worlds. The pessimist fears that this is true.

How well will their wishes align with the interests of the rest of us?

Very well, in my opinion, if you look at OpenAI's and Anthropic's mission statements, labor market impact research, and extensive talks on the matter.

Yes, because big tech has such a great track record of sticking to their ethics over profits. None of them have any sort of history of dishonesty. Their consistently great politics and world outlook is so reassuring too. /s

No it’s not vanishingly small. Every intelligent being knows not to hurt others for no gain. If your first instinct when seeing an animal is to kill it then you are a bad person.

You are secretly worried AI would be indifferent to you

>Every intelligent being knows not to hurt others for no gain.

As the article clearly explains, an ASI has plenty to gain by killing you. You cannot help it, but you can potentially hinder it, so the optimal choice is removing the risk.

Reply on news.ycombinator.com