Hacker News Reader

Comment by OkayPhysicist | Hacker News Reader

The writers for Star Trek spent a lot more time thinking about interstellar travel than I do, but I'm not going to go to them for advice about building a rocket ship.

The Rationalists are a philosophical/pseudo-religious organization in the same way the Daoists are. Just spending a lot time thinking about something doesn't make them right, it just makes them internally consistent with their chosen axioms. That is useful: if you agree with the axiomatic beliefs of such a movement, you can probably defer a lot of reasoning about the conclusions of those beliefs to the authority figures within that movement. But it doesn't change the probability of the axioms themselves being true, any moreso than the Pope saying so affects the probability of God existing.

AI Is Important And Potentially Dangerous is an axiomatic belief in the rationalist circles that are relevant to AI safety, in the same way that God Exists is axiomatic to Catholicism.

View on HN

That's a fascinating assumption, and I wonder how you got to it, because it doesn't match anything I've ever seen. Most groups look more homogeneous from the outside than the inside. You are assuming their belief is religious and axiomatic, rather than very carefully reasoned about for decades, iterated on, discussed, debated, tested, and refined.

"Most groups look more homogeneous from the outside than the inside."

But that doesn't mean that they really are all that different in the end. It reminds me a of joke as told by Emo Philips that I think is quite apropos:

==-==-=

A man was walking across a bridge when he saw someone about to jump. He ran over and said, "Don't do it! There's so much to live for!"

"Like what?" the person asked.

"Well, are you religious?"

"Yes."

"Me too! Are you Christian or Jewish?"

"Christian."

"Me too! Protestant or Catholic?"

"Protestant."

"Me too! What denomination?"

"Baptist."

"Me too! Northern Baptist or Southern Baptist?"

"Northern Baptist."

"Me too! Northern Baptist Conservative or Northern Baptist Liberal?"

"Northern Baptist Conservative."

"Me too! Northern Baptist Conservative Covenant or Northern Baptist Conservative Council?"

"Northern Baptist Conservative Council."

"Me too! Northern Baptist Conservative Council of 1879, or Northern Baptist Conservative Council of 1912?"

"Northern Baptist Conservative Council of 1912."

The man stepped back, eyes narrowing, and yelled, "Die, heretic scum!" and pushed the other off the bridge.

It's good game theory. You're much more likely to be able to bully a Northern Baptist Conservative Council of 1879 members into joining your cult than anybody else.

> Most groups look more homogeneous from the outside than the inside.

The converse is also true: Most groups look more heterogeneous from the inside than the outside. Again, to use the Christianity example: it takes a fair bit of inside baseball to figure out how various different sects and denominations are any different, but if you ask one of their priests, they'll be sure to tell you how Baptists and Episcopalians couldn't be more different.

It's a symptom of normalizing to an Overton window. From inside a church the belief that Jesus was just a guy who talked to God might seem an extreme fringe belief to someone who believes that Jesus is fully God, but from the outside we can lump them together as axiomatically holding that God Exists, because we have an Overton window ranging from "God doesn't exist" to "Allah talks to me everyday, and she gave me my promotion".

That not exactly a withering criticism of the rationalists because most people who have any opinion about AI believe that AI is important and potentially dangerous.

Reply on news.ycombinator.com