Comment by AndrewKemendo | Hacker News Reader

Comment by AndrewKemendo | Hacker News Reader

Can someone give an alternative vision of the next 100-200 years that doesn't include AGI/ASI? I don't see it. At least one that isn't a nuclear wasteland.

I have thought about this every day for 32 years and don't know of any possible alternative future than one run alongside or by Artificial Superhuman Systems. I just can't possibly look into the future and expect the current state of machines to plateau here and stop. In fact that's my life's work and has been so since I was 13.[1] I even discussed this at the AGI conference in Laval in 2014 with Turing winner Yoshua Bengio [2]

The Overcoming Bias blog, which turned into Less wrong, was the best place for these discussions until it became a cult of personality around 2013. The SIAI and all the mess that it became at least was a chance to discuss it. There's no serious place to discuss it.

AGI discussion went from nerds that everyone pointed and laughed at, to now being an imminent existential threat "To the average person, this all sounds like the plot of a sci-fi novel. There might be a tendency to dismiss a techno-utopia as "billionaire boys and their toys" but Torres says the danger comes from the amount of power and influence these billionaires have to make policy and spend money."

There have been, at least a dozen core books from Bostrom, Kelly, Kurzweil, Shannon etc... that have discussed this concept in precise and exacting detail.

Arguably the whole reason to make computers has been to make AGI/ASI since the first mechanical turk etc...

So I'm not really seeing a future without ASI and despite a lot of talk inside computing - for the entire history of computing - it's happening now, and everyone is surprised for some reason.

[1]Here's my position in more detail:

https://kemendo.com/Beyond-Human-Limits.html

[2] https://www.youtube.com/watch?v=exhdfIPzj24

AGI/ASI as we think of it in science fiction is not guaranteed. I think its definition will erode due to marketing until it's a label on a product, but less limited than what we think it could be today. It'll probably be a capable system but not able to do science on its own.

I don't think we'll throw nukes. AI will have taken care of work that requires text and 95% of us are left doing manual labor because a robotic workforce still isn't feasible. US/Canada will start moving its cities north as the planet warms, and we have to start new farms up there.

That’s not an alternative

And in fact the second world you describe is precisely the world with artificial superhuman intelligence

Your link is broken

To be fair, we don't know the future. Solving scientific problems have eluded researchers and their billions of dollars for centuries. Genuine AGI will be on par with solving P=NP or curing cancer. Its a profound problem that gets to the very root of what we are.

100 years from now, AGI could be viewed as such an improbable proposition as flying cars, teleportation, ghosts, or faster than light travel.

Which link?

Here’s one: https://kemendo.com/Beyond-Human-Limits.html

I agree we'll have AGI/ASI in the next 100-200 years, probably the next 10.

I'm not sure about "Humans will not exist in a million years from today" in [1] though. We've been quite good at preserving interesting species.

Re. places to discuss it - x/twitter? There are serious people on there though it's not easy to chat. r/singularity is quite jolly but not a very serious place.

https://kemendo.com/Beyond-Human-Limits.htm gives me a 404

fixed now, thanks

Eh. I can rather trivially imagine a future where AGI development gets interrupted by a series of domestic terrorist attacks in response to some critical mass of economic disruption being hit. Folks get cranky when they're having difficulty feeding their kids, and given how comprehensively armed the US populace is that's bound to get spicy.

That just falls into the second category (violent reaction) where it’s specifically interrupted

it doesn’t give an alternative society that never creates it

I'm not sure I agree there. You don't need to hit anything like AGI to brick the bulk of the economy. 60% of the US works service industry jobs that could be replaced with a kiosk and automation that requires zero problem-solving ability. I also don't think AGI is possible, but that's a separate discussion.

> Can someone give an alternative vision of the next 100-200 years that doesn't include AGI/ASI? I don't see it. At least one that isn't a nuclear wasteland.

Eh? Without artificial super intelligence the future is a nuclear wasteland? How does that follow?

Why wouldn't an artificial super intelligence conclude that humans are the problem and nuke the lot?

My claim is that the only world that doesn’t make AGI/ASI is one where humanity is wiped out first

People could always choose not to make it. That was Frank Herbert's idea in Dune with the Butlerian Jihad:

https://dune.fandom.com/wiki/Butlerian_Jihad

> Can someone give an alternative vision of the next 100-200 years that doesn't include AGI/ASI? I don't see it. At least one that isn't a nuclear wasteland.

I don't think I can. I can only see a post-apocalyptic landscape similar to mad max, terminator or the fallout series.

If we look at the actions of these AI leaders more than what they are talking about in-front of the camera it does not look like a utopia for us.

If Zuckerberg, Sam Altman and Larry Ellison are building bunkers, I guess that they have a backup plan when there is societal collapse when AGI accelerates into total job displacement.

It's a utopia for billionaires that have power, influence and lots of spending power; unaffected by AGI. They mention that a theoretical UBI system will solve the issue with job displacement which is ill thought out and has never worked sustainably at a large scale.

Of course there will always be people that will try to slow it down, but the ultimate endgame is full job displacement + humanoid robots taking over manual labor with no alternatives for those lost jobs.

Reply on news.ycombinator.com