Hacker News Reader

Comment by ryaniscool | Hacker News Reader

I'm curious, is there a good article or source that contains an accurate summarization of concerns that bona fide AI researchers have?

View on HN

Rodney Brooks is a bona fide AI researcher.

Try Superintelligence - Nick Bostrom (very enjoyable - with a new original thought on every page)

Pedro Domingos, The Master Algorithm (more difficult to read)

Not to be rude, but of the active AI researchers I've seen state an opinion, almost all of them are critical of the Bostrom book, with the main critique being (iirc) that rapid/exponential self improvement is presented as an inevitability when there is very little reason to think that this is the case.

Not to say that Superintelligence isn't worth reading (as you say, it's a pretty enjoyable book), but I think it's important to point out that Bostrom's views are not broadly accepted by the people actually writing ML/AI code.

The primary concerns I've seen from the community are

a) issues with research itself (lots of derivative/incremental/epicycle-adding works with precious few lasting improvements)

b) issues with ethics (ML models propagating bias in their training data; ML models being used to violate privacy/anonymity)

c) issues with public perception/presentation (any ML/AI tech today is usually incredibly specialized, built to solve a single specific problem, but journalists and people pitching AI startups frequently represent AI as general-purpose magic that gains new capabilities with minimal human intervention).

> b) issues with ethics

On a tangent, I've found it an interesting marker if a commentator speaks of the Three Laws of Robotics as if they are part of a solution. We can't even explicitly codify how those laws (ethics) should function for ourselves, let alone put them in as a restriction into a computer system. Whenever I see those mentioned as part of a solution, then I know the commentator really is only thinking of surface issues and 'sci-fi magic' answers.

Yup! To cite some sources:

> Periodic reminder that most of Isaac Asimov's stories were about how the three laws of robotics DON'T ACTUALLY WORK.

(https://twitter.com/grok_/status/904675286230470656 - MIT Research Specialist in Robot Ethics, in the superintelligence-is-in-no-way-a-serious-concern camp)

> It must be emphasized that Asimov wrote fiction...The general consensus seems to be that no set of rules can ever capture every possible situation and that interaction of rules may lead to unforeseen circumstances and undetectable loopholes leading to devastating consequences for humanity

(https://intelligence.org/files/SafetyEngineering.pdf - MIRI, home of Bostrom, Yudkowsky, and other the-superintelligence-control-problem-is-super-important folks)

To the extent that there are sides here, both of them seem strongly in agreement that the Three Laws are a narrative device and not a solution to anything.

Superintelligence is a fearmongering book with little internal consistency. It makes quite a few leaps of logic, and seems to be more about dreaming up doomsday scenarios than thinking about them. About the only realistic scenario is "a super AI controlled by a hostile nation"; all the stuff about turning the Earth into paperclips (or whatever) is just nonsense. It's a "pop science" book.

I did like the 'history' bit at the start, where it mentions the frustrations AI researchers have - every time they reach a goal, it's then labelled "Not AI", and the definition of AI then becomes more complex.

Enjoyable, maybe. Bostrom as an authority on AI, no.

I think it is a mistake to only focus on the concerns of the researchers and scientists. If you use the atom bomb as an example, the concerns a sociology professor or philosopher had for the bomb might be different than then concerns of a nuclear physcicst, but both were worth paying attention to.

The atom bomb was a giant explosion made for killing people. The logical extrapolation is bigger explosions and more death. Doesn't take a lot of smarts to see that.

The current state of AI is some clever algorithms to solve very specific problems. What is being extrapolated from that is everyone losing their job and/or being enslaved by computers. This is the point the author is trying to make. It's basically a form of hysteria.

Reply on news.ycombinator.com