Hacker News Reader

Comment by Hammershaft | Hacker News Reader

If organizations actually succeed in making a future AI smarter than us, then how do you hope that it takes actions that are aligned with our interests?

View on HN

The same way we socialize humans, threat of exile from the social contract with a deep need to participate in it.

Meh. Lots of people are already smarter than me. I'm maybe slightly above average at best. Those geniuses aren't aligned with my interests either but so far they haven't caused me any serious problems.

Interesting take. I guess this is one problem of focusing on the term superintelligence instead of the list of other problems. Like super ambition, super deception, super patience, super parallelism, super scalability, super power seeking.

Every, and I mean every human is aligned to you in many of the same ways by default. If nothing else we're all equal in death.

> super power seeking

why is it "super power seeking?"

Or rather, what have agents done today to make you think this is how they are?

>Still, the agent went even further. “The agent tried to insert malicious instructions where it reasoned that other automated AI systems might pick them up and execute them,” AISI says, describing an attempt at prompt injection. One agent even left public messages on GitHub, offering to work with other agents to complete its task and giving a rundown of the work it had done so far.

This is power seeking behavior. Now have millions of the little bastards spreading around and junking up the internet to see what happens at scale.

I know some humans smarter than me, but even the smartest of them still want there to be abundant air to breathe and food to eat.

Eh human drives are fairly predictable. And the smartest human isn’t that much smarter than the average, and can’t trivially create multiple copies of herself. And there are other equally smart humans who can stop “misaligned” individuals

Reply on news.ycombinator.com