Satori in 3 ... 2 ...
Sorry, it had to be done.
http://kruel.co/2013/01/12/rokos-basilisk-everything-you-nee...
The parallells to religion should be obvious to anyone with the ability to see the bigger picture, but it's somehow reassuring that even the most rational of humans can transform into a religious nutjob and not realize it. The road to hell is paved with good intentions, etc. :-)
Frankly, the article is still insane. (Look at the talk page for my girlfriend's reaction. She felt like throwing my laptop across the room for wasting a second of her time on this inanity.) This is because this shit is almost impossible to explain to normal humans. I have found it easier explaining Scientology to normal humans.
Pretty much no-one on LessWrong endorses or believes in the basilisk ... they just advocate most of its component parts.
LessWrong are generally a pretty good bunch, but this will be their Xenu. Unfair as that may be. And they did it to themselves.
(gwern - if you're wondering why Dmytry can be quite such a dick at LW about this stuff, it's because he gets email from LW's distressed children too. He does his best to be a helpful, decent person.)
They use the non-controversial value-add stuff to draw new members into the cult. Eliezer has stated in interviews that the most important thing you can do is help him on his research if you can, and otherwise to make as much money as possible to give to his research, save a bit for your living expense. Now he has dozens of people that agree with him completely, a few hundred that mostly agree with him, and thousands that think he's a pretty swell guy and don't realize the shady depths of the philosophy he's pushing.
I'd much rather people change their mind as they gain greater understanding, as long as they document the reasoning.
Eliezer often cites a theorem from a mathematician that says that ideal rational agents can't disagree and applies it to normal conversation. In the LW circle this is used to bully people into conformity. No allowance is given for the time that it takes humans to think things through or negotiate on a set of priors based on their joint life experience. In practice at LW, it means that if you haven't conformed to a result that disturbs you within the course of a single conversation then you must be irrational. Because by Aumann's theorem rational agents will not remain in an undecided state.
They really are a bunch of borg.
On the other hand, I see a lot of examples of people being unusually polite and curious about people who disagree with them, and who try to specifically cultivate the ability to drill down to the root cause of arguments and learn as the result of a dispute.
It has some questions that do a reasonable job of surveying people's beliefs on transhumanisty and Yudkowsky-y stuff like AI, cryonics, and many-worlds. Going by the survey, the population is more heterogeneous than you give it credit for.