Comment by AndrewKemendo | Hacker News Reader

Comment by AndrewKemendo | Hacker News Reader

Parent Full thread

Certainly I am not the first to describe the behavior of a non-trivial number of outspoken members of the community as "cultish." In large part my complaints stem from:

1. Dogmatic adulation of Yudkowski as a "savior" figure.

2. An almost immune response resistance to "practical" approaches to the Friendly AI problem. For example, laying out even a basic path to the structure of one - given that's their mission.

3. In light of #2, the community's serious consideration of un-testable scenarios like Roko's Basilisk

4. A flavor of authoritarianism, racism and sexism from former founder as Michael Anissimov.

5. Critical lack of diversity in backgrounds and approaches to ML/AI, which keeps the thought process fairly insular. For example you never see these people talking/attending AI conferences, and I'm not sure I have ever seen them at the AGI conference. AFAIK also they only publish in-house with liberal internal reference.

Those are some from the top of my head. Granted I think they have sorted some of this stuff out, like #4, but all of it together makes for a toxic and fragile community, largely built around a singular ego.

I haven't been keeping up with them as much in the last few years so some of these things might be getting better - but when I left the community in 2013 these were my main problems.

> 1. Dogmatic adulation of Yudkowski as a "savior" figure.

They're just copying the man himself, who sees himself that way. But before you dismiss that as cultish, be sure you fully understand the problem he's trying to solve and how instrumental he has been to solving it so far (in terms of raising money, convincing people, building a community, knocking down bad arguments, etc.).

> 2. An almost immune response resistance to "practical" approaches to the Friendly AI problem.

What you have to bear in mind is that there are plenty of "practical" approaches to the Friendly AI that are simple, elegant - and wrong. Have some humility and note that if it was that easy, someone would have published a peer-reviewed solution already and that would be that.

> In light of #2, the community's serious consideration of un-testable scenarios like Roko's Basilisk

The community rejected it (with help of a site I contribute to, RationalWiki), and probably would have rejected it faster and more decisively if Yudkowsky hadn't unsuccessfully tried to censor all discussion of it.

> A flavor of authoritarianism, racism and sexism from former founder as Michael Anissimov.

Anissimov is indeed a straight-out fascist, in my view - but that just shows bad judgement on the part of Yudkowsky, that should not be used to tar the entire community with the same brush.

> Critical lack of diversity in backgrounds and approaches to ML/AI, which keeps the thought process fairly insular. For example you never see these people talking/attending AI conferences, and I'm not sure I have ever seen them at the AGI conference. AFAIK also they only publish in-house with liberal internal reference.

They are starting to publish in mainstream venues but most of their output has been online so far. Also, this stuff is hard, it's not like your typical "add another epicycle, cite self again" stuff of a lot of scientific research. And they are not trying to maximise the number of papers they publish, but rather trying to make progress on a specific research agenda. The former objective is not the same as the latter and could even impede the latter.

Reply on news.ycombinator.com