Hacker News Reader

Comment by 13years | Hacker News Reader

Potentially there are no good outcomes even if AGI remains under control. If we are actually able to create the type of AI entity they are trying to create, one that exponentially improves, the implications are beyond what most have thought about.

Something I recently wrote in depth about here. How we completely misunderstand the future that we think will happen.

"The implication is that everyone is enthusiastically racing towards a destination that does not exist. The capability to make the things you want will ironically be the same capability that makes them unattainable. This is not a scenario that arises from some type of AI failure, but rather this is assuming AI performing exactly as intended."

https://www.mindprison.cc/p/the-technological-acceleration-p...

View on HN

> https://www.mindprison.cc/p/the-technological-acceleration-p...

That is a very good and insightful article. One of the biggest issues I have with AI enthusiasts is that they talk about creating these incredible capabilities ... then they imagine the resulting world with them will look like some hackneyed sci-fi work, like Star Wars, whose universe seems like it could be kinda cool to live in until you actually think about it and realize it's all fridge logic and nothing about it actually makes sense (e.g. WWII-style dogfights in the vacuum of space).

The only issue with the article is sometimes it's too optimistic, e.g.

> No one will care about you or what you create as there is nothing you can offer they can’t simply wish for themselves. Life is all about you and the machine, nothing else. We land in a world dominated by techno-dystopian narcissism.

> ...

> Whatever may be the current direction of society under the present technological capabilities, we can only assume AI will accelerate society towards the path we currently are traveling which is toward a techno dystopia of a populace mesmerized by shiny glimmering lights.

I kind of find it hard to believe there would be any stability for any people suck in that state, I think they'd instead be "cleaned up" and disposed of in relatively short order. Those people would be utterly useless to whatever still has power in society, so why service and protect them indefinitely?

>That is a very good and insightful article.

Thank you!

> The only issue with the article is sometimes it's too optimistic, e.g.

The reply below is criticizing for being too pessimistic, lol :-)

I agree, it might be only a transient state. We really can't comprehend how we would deal with such existence if at all. Would we manage to keep our sanity?

Nonetheless, I don't claim to have the right answers, just questions and perspectives that I hope inspire deeper thought about the journey we are travelling and hopefully deeper reflection as to whether it is the journey we want to be on.

What I do find interesting, are those that read it and still don't perceive it at all. Their response is "That is exactly what I want." I suppose that is the case as was in an excellent allegory for this "A Nice Place to Visit" Twilight Zone episode. It plays out this scenario very nicely.

> What I do find interesting, are those that read it and still don't perceive it at all. Their response is "That is exactly what I want."

I think there's a basically (new movement) religious desire to see a lot of these technologies come to fruition, either radically fundamentalist or lazier and less informed, that interferes with the application of the implications. They believe in the idol of technology, as told in the good news of sci-fi, and avert their eyes from the glaring cracks, blemishes, and lifelessness of it.

I sense a lot of things in those types that I can't fully articulate, but some of the things they say are just mind-boggling, and gesture at a broken and unhinged aspect of software engineering culture.

Yes, I agree that is some component of it as I have felt the same vibe from some.

Reply on news.ycombinator.com