Hacker News Reader

Comment by Kiro | Hacker News Reader

And it boggles my mind that you confidently make such extreme claims.

Why? Every day the rest of the world make confident extreme claims that AI/advanced technology/STEM will improve the world with zero evidence either, and no one ever challenges those. In fact, everyone just jumps on the happy "tech is the best" bandwagon with nary a thought. If you think I'm making extreme claims, go look at the one-sided only positive view of technology being espoused by everyone else.

On the other hand, technology has brought us to the brink of a climate disaster and a screwed up society (just look at how much research there is for the damaging effects of social media).

So you've realized that one side of the extreme is irrational because it's extremely over-positive, and decided you'd just jump on the other side of the irrational extreme over-negative to be contrary? Saying AI will save the world is just as irrational as saying it will destroy it. By your own admission, even.

Well, I don't think my statements are extreme actually. I've given them a lot of thought and reasoned them out more fully elsewhere. I do sincerely believe AI has the capability to make society significantly worse and even destroy it through its isolating effects and through its removal of purpose from humanity. Of course, these comment boxes are a little small to have extended discussions but I'm perfectly willing to hear any serious arguments against what I have to say.

Do you have a blog or website with your thoughts on this?

Yes I do. I'm not sure it's allowed to link here but it's blog.jpolak.org. Some posts are quite long with much more detailed analysis and arguments.

I also recommend David Skrbina's "The Metaphysics of Technology" although my posts and thoughts are independent of that book.

The modern development of AI can be likened to the earlier development of the atom bomb. Everyone thinks it will destroy the world because technically it has the potential to do so and everyone's a pessimist on average. But it'll probably not be nearly as bad as claimed. Us humans have a strong tendency to enjoy being alive and preserve doing so.

The atom bomb is an extreme device, the issue is the extremeness is multi-polar.

Atom bombs have extreme explosions, but they also require extreme maintenance, extreme precision in manufacturing, and are extremely detectable.

For AI the level of extremeness has not been defined. Yea, it requires extreme manufacturing, but it's the same manufacturing that builds billions of chips per year that are distributed almost everywhere. A server farm running AI looks much the same as a server farm running other computing tasks.

Coming back to atom bombs, yes, atom bombs could destroy the world 10 minutes from now, and every 10 minutes from now until we destroy enough of them they can no longer do that. It simply has not happened yet and we've likely been very very lucky that it has not happened.

Lastly, AI, especially when we reach AGI, is intelligence which means it will likely have some hand in if the world is destroyed or not. This risk rapidly increases if AGI capabilities are far beyond human capabilities.

Atom bombs are just tools, they are only controlled by humans. But a super intelligence can be an actor, not just a tool. This changes everything. It might be vastly more intelligent than humans. It wouldn't be limited by skull size, meager energy provided by food, or low spike frequency of biological neurons. The difference might be as large as between humans and chimpanzees, or humans and mice, or even humans and ants. There is no reason to expect we are anywhere near the theoretical peak of intelligence.

Intelligence is extremism. It is the most potentially extreme agent in the universe because it allows you to control all other forms of extremes.

When humans intelligence grew beyond that of the other animals, we became the masters of those animals. We can choose to manipulate them, control them, or if they cannot be controlled, destroy them.

Now, I don't understand why saying "There is no particular reason why humanity is the highest form and capability of intelligence" would be an extreme claim in any way. At least to me, the opposite is true "Humans are the most capable form of intelligence possible" is a anthropocentric and extreme statement.

Reply on news.ycombinator.com