Ask HN: Is misaligned AI less dangerous than AI controlled by a bad actor | Hacker News Reader

Ask HN: Is misaligned AI less dangerous than AI controlled by a bad actor

2 pointsInfiniteRand5 comments

Ultimately I think, most trains of “thought” that could be a milestone towards a misaligned AI taking a dangerous action could be input by humans, especially if programmers have little oversight on what they put into their AI systems. An analogy that occurs to me is a plane, a powerful body of technology that can ultimately be controlled by a single person if they try. We have seen many cases where bad actors have taken advantage of plane technology and set it on a destructive path, either for political goals or suicidal intentions. I suppose in theory a misaligned AI might find it easier to hide its tracks but then again a bad actor could instruct the AI to hide its tracks

View on HN

I think a misaligned AI is vastly more dangerous than a bad actor. Bad actors are statistically much rarer than regular users, so it really comes down to scale. Let's say we have a misaligned AI, and it naturally does bad acts. That means nearly every regular user can become a bad actor via the AI's actions. But if the AI is aligned and the bad actor just jailbreaks it, sure that's bad, but not nearly on the scale of a misaligned AI.

I think AI controlled by a bad actor is a more immediate risk, while misaligned AI is a potentially much larger long-term risk

No one can guarantee an AI won't hallucinate. The real danger is how overconfident they get. Bad actors would wait for AGI

I'm not saying anything particularly interesting here, but alignment in the limit isn't a solvable problem, you basically just trade control risk for concentration of power risk.

If an AI is aligned to always listen to humans, then AIs could, if asked, be used to facilitate all kinds of horrendous things that humans ask of it.

Even with our current AIs we understand that AIs trusting humans completely would be a disaster so they are designed to refuse to do what you ask of them if it believes you might have bad intent. It may even report you if you disclose something to it in confidence that it believes is wrong.

So we want to give AIs some sense of moral judgement, and we want them to the right thing in any given scenario, but this mean we need to allow them to make moral judgements and execute on them in ways we may not agree with. If I try to harm another human then an AI is probably right to stop me, perhaps with lethal force depending on what I am doing.

The problem is we humans don't agree on right and wrong, and nor will AIs. For example, if WWII broke out today would the right thing to do be for the AI to help the Allies and kill the Germans, not participate in the war, or take control and force the humans to stop fighting and killing each other? Clearly it's the third, and that's a problem.

100%. The idea of AI alignment is an oxymoron honestly. Humans are not aligned with each other, so how exactly do we align it to something that doesn't exist?

Its like being given the task of "choosing the universally agreed best colour of the rainbow" kinda misses the whole point of a rainbow. Same with AI the whole point is that its probabilistic not deterministic.

Reply on news.ycombinator.com