Hacker News Reader

Comment by fudged71 | Hacker News Reader

Best I can find: http://www.pbs.org/newshour/bb/business/july-dec02/google_11...

SPENCER MICHELS: Sergay Brin thinks the ultimate search engine would be something like the computer named Hal in the movie 2001: A Space Odyssey.

SERGEY BRIN: Hal could... had a lot of information, could piece it together, could rationalize it. Now, hopefully, it would never... it would never have a bug like Hal did where he killed the occupants of the space ship. But that's what we're striving for, and I think we've made it a part of the way there.

View on HN

Funny, because Hal didn't had a bug. It did what it did interpreting the orders that humans gave it. Just unintended consequences.

Which is kinda the point about AI safety. We don't really know what we want, we sure as hell don't know how to formalize it, and we're lucky that we don't know how to create a real AI because otherwise it could surprise us in a very ugly way, just lawfully following the plan we gave it.

"hopefully"

Oh. Not rules then? Well, what rules. How about giving it a rule "children must not die". Apparently we're big on that one. Except, wait, we mean "US children must not die", so we can still bomb foreign suburbs where children are known to exist. But then we'll need "children must not die on US soil" because it is still possible that the union of (isa US minor citizen ?x) and (isa-resident baghdad ?x) is non empty and the AI would be unable to act. Of course then we need, "US children must not die except in a car", because otherwise the AI would have to destroy cars.

Ok, so lets abandon rule based AI altogether and model our AI on biological processes like humans. Well humans would never cause mass suffering of other humans. We have laws against it. Oh.

"hopefully"

"Let's play global thermonuclear war!"

The only winning move is not to program AI?

Here's an interesting discussion about AI, rules, morality, etc. http://www.reddit.com/r/Futurology/comments/y9lm0/i_am_luke_...

Reply on news.ycombinator.com