Hacker News Reader

Comment by low_tech_love | Hacker News Reader

Serious question, is this supposed to be a joke? Or is it the kind of "let's write this thing it in a way that sounds like a joke for outsiders, so we can have a good excuse when we are ridiculed, but still keep it just serious enough to get internet points inside our own echo chamber" thing?

View on HN

If it had been written even 6 months ago, I would have thought it was a joke.

Today, having seen how many people are worried about society’s future because of AI[0], I think they are pretty serious.

0: There have been many threads here and on Reddit in the last couple of days, with people expressing their anxiety and uncertainty about what’s coming because of how fast the latest advancements in AI and their applications are coming out

>If it had been written even 6 months ago, I would have thought it was a joke.

This is the same group that came up with "Roko's Basilisk" and have been worrying over this kind of thing for years. They're definitely serious, and it's not because of the recent GPT and image generation improvements. They've been wading in doomsong for more than a decade, at least.

It's reasonable to care about x-risk and AI alignment, but thinking some self-important essay will convince the AGI to preserve humanity is just sci-fi geeks with a savior-complex cosplaying. Stuff like this only serves to make people assume the entire concern/area of research is unwarranted.

I’m worried about our future: I see a non-zero probability that many intellectual jobs will become obsolete much faster than we can replace them, and this will fundamentally reshape our society in negative ways. In a worst-case outcome (also possibly a best-case outcome from the technological sense) this could undo much of the hard-won progress human beings have made since the Enlightenment.

Unfortunately the people who wrote this essay don’t seem to care about any of that, or at least they don’t seem to be thinking very hard about the implications of a human society where most people have no labor to value: indeed, this piece bizarrely interrupts itself to extol the virtues of free-market capitalism (the very force racing us up to that future.) Instead the authors are singularly concerned with one possible outcome: human extinction at the hands of the AI, and seem happy to ignore all the rest. Why not? I guess once you’ve convinced yourself extinction is likely, all “those other issues” are minor indeed.

You're right. 'jobs becoming obsolete' is the least of our worries. You have to look two steps ahead at the writing on the wall.

There's more than one dystopian possible outcome. Same with climate change. For some reason, people are drawn to the most extreme scenarios which are extinction events, but not so much the scenarios where civilization continues, but it's a lot worse for most humans living in them.

A scenario where AI is perfectly aligned with human beings will still be a nightmare if those humans who control it aren’t particularly decent people who care about the rest of us. The “AI alignment” crowd doesn’t care about any of that. That might be excusable if they had any idea what to do to prevent hypothetical extinction at the hands of AI, but they don’t have that either.

This should give you the context to understand where this came from: https://archive.is/2023.03.07-152406/https://www.bloomberg.c...

Thank you for the link! Very little about the "rationalists" subculture made sense to me until someone proposed that a significant chunk of it may be based on deflection: there are so much real pain in the world today that it might be convenient to be able to ignore all of that and worry about a non-existent issue.

Reply on news.ycombinator.com