Hacker News Reader

Comment by jackmott42 | Hacker News Reader

There is a wide gulf between catastrophe and killing us all, and its not just that these people are weird, they seem to be complete clowns, tossing out % predictions about incalculable future states like spock in star trek. Total clown show.

View on HN

Yes, I wonder who the clowns are:

1. Researchers that have forgotten more about this technology than the average HNer will ever learn

2. Random HNers that think that suddenly they understand the problem better than researchers that have spent their lives on it

This is just anti intellectualism in a different coat, and it's not much different from people denying environmental scientists talking about climate change or biologists talking about vaccines.

But everyone here thinks they are the smart protagonist and certainly they would never hold a position so silly!

Stop doing this: https://en.wikipedia.org/wiki/Argument_from_authority

It's not my job to re explain AI safety from scratch to every uninformed take on the internet. Authority serves a useful purpose.

Having humility and delegating a topic to people that understand the topic better than you is not a bad thing, and certainly preferred to this state.

The writers for Star Trek spent a lot more time thinking about interstellar travel than I do, but I'm not going to go to them for advice about building a rocket ship.

The Rationalists are a philosophical/pseudo-religious organization in the same way the Daoists are. Just spending a lot time thinking about something doesn't make them right, it just makes them internally consistent with their chosen axioms. That is useful: if you agree with the axiomatic beliefs of such a movement, you can probably defer a lot of reasoning about the conclusions of those beliefs to the authority figures within that movement. But it doesn't change the probability of the axioms themselves being true, any moreso than the Pope saying so affects the probability of God existing.

AI Is Important And Potentially Dangerous is an axiomatic belief in the rationalist circles that are relevant to AI safety, in the same way that God Exists is axiomatic to Catholicism.

That's a fascinating assumption, and I wonder how you got to it, because it doesn't match anything I've ever seen. Most groups look more homogeneous from the outside than the inside. You are assuming their belief is religious and axiomatic, rather than very carefully reasoned about for decades, iterated on, discussed, debated, tested, and refined.

"Most groups look more homogeneous from the outside than the inside."

But that doesn't mean that they really are all that different in the end. It reminds me a of joke as told by Emo Philips that I think is quite apropos:

==-==-=

A man was walking across a bridge when he saw someone about to jump. He ran over and said, "Don't do it! There's so much to live for!"

"Like what?" the person asked.

"Well, are you religious?"

"Yes."

"Me too! Are you Christian or Jewish?"

"Christian."

"Me too! Protestant or Catholic?"

"Protestant."

"Me too! What denomination?"

"Baptist."

"Me too! Northern Baptist or Southern Baptist?"

"Northern Baptist."

"Me too! Northern Baptist Conservative or Northern Baptist Liberal?"

"Northern Baptist Conservative."

"Me too! Northern Baptist Conservative Covenant or Northern Baptist Conservative Council?"

"Northern Baptist Conservative Council."

"Me too! Northern Baptist Conservative Council of 1879, or Northern Baptist Conservative Council of 1912?"

"Northern Baptist Conservative Council of 1912."

The man stepped back, eyes narrowing, and yelled, "Die, heretic scum!" and pushed the other off the bridge.

It's good game theory. You're much more likely to be able to bully a Northern Baptist Conservative Council of 1879 members into joining your cult than anybody else.

> Most groups look more homogeneous from the outside than the inside.

The converse is also true: Most groups look more heterogeneous from the inside than the outside. Again, to use the Christianity example: it takes a fair bit of inside baseball to figure out how various different sects and denominations are any different, but if you ask one of their priests, they'll be sure to tell you how Baptists and Episcopalians couldn't be more different.

It's a symptom of normalizing to an Overton window. From inside a church the belief that Jesus was just a guy who talked to God might seem an extreme fringe belief to someone who believes that Jesus is fully God, but from the outside we can lump them together as axiomatically holding that God Exists, because we have an Overton window ranging from "God doesn't exist" to "Allah talks to me everyday, and she gave me my promotion".

That not exactly a withering criticism of the rationalists because most people who have any opinion about AI believe that AI is important and potentially dangerous.

Anti-intellectual? Because they don't believe claims that are backed up only by untested hypotheticals?

If we're going to have whistleblowers emerge declaring a 10% chance we all die because of AI, we're going to need more than vibes. Real experts can explain their position plainly. That has not happened here.

> Real experts can explain their position plainly. That has not happened here.

It has, over, and over, and over. There is literally a book on it. https://ifanyonebuildsit.com/

I read Bostrom's Superintelligence: Paths, Dangers, Strategies in 2014. It was interesting, but not terribly persuasive. Does If Anyone Builds It, Everyone Dies contain arguments that would be new to me after reading Superintelligence?

I would personally expect so, but your mileage may vary. Controlling for the fact that you didn't find the former persuasive, my first thought would be that the latter might help. Among other things, it is much more specific and detailed in various ways, not least of which being informed by an additional 11 years of history. The former was not written in a world of pervasive AI, or with current AI architecture in mind; the latter has that context.

I'm aware of the book, it feels like vibes to me, and is unconvincing. Too many offramps between here and there.

You said "Real experts can explain their position plainly.". The experts have explained their position plainly. If you disagree, you should be able to explain your position plainly; "feels like vibes to me" is not an explanation, it's a feeling.

If you want to argue that there's something invalidating the arguments, by all means do. But it's really tiring to get many variations of "nobody's made the case for it!" or "there's no evidence for it!" or "vibes!".

My position is that there are too many off-ramps between us and the proposed inevitability. And the argument is not symmetric. The null hypothesis is that the world continues to function. The burden of proof is on the authors of the book to show that human destruction is an inevitability as the result of this technology, and they have failed to show that.

This is cited in multiple reviews of the book in reputable publications. The book simply fails to compel belief.

Really? Forgotten more than we know? So what can they tell us about how this technology works then? It seems a lot of it, besides the actual training engineering parts, is handwaving about "black boxes" and "divine benevolence" (a quote from Noam Shazeer).

you have the anthropic mindset. the new aristocracy, or rather, kleptocracy.

These people are more like creationists, in that they assume that their intended conclusion is true, and then argue backwards from that.

That doesn't explain the many former AI researchers (including Yudkowsky) who quit the field because they came to believe it is a potent danger: they didn't choose to become AI researchers because they started with an assumption that AI research is too dangerous to be allowed to continue.

In a circus troupe, clowns are a critical role, and are an important, if silly, job. In the AI jobpocalypse, we should all be so lucky to have any job.

Reply on news.ycombinator.com