Hacker News Reader

Comment by kurthr | Hacker News Reader

I almost didn't click on this... it turns out the scientists are just as depressed by the news. I'm not surprised that 1.5C is overly hopeful. Frankly, it looks like geoengineering may be more viable/profitable/necessary than currently discussed.

View on HN

I think it will come like most things - as a sigmoid curve. People will ask for it once the catastrophes become locally constant.

Makes me truly wonder if we wouldn't be better off with a benevolent AGI controlling things on our behalf

That's just creating another entity to shift the burden of decision to. Problem is, even if we had an AGI, it's not God. It's trained on all the dumb things humans have said and written, so I don't see any reason why it should automatically be smarter.

My hope is that an AGI would be able to differentiate between “dumb things humans have said” and physics / real data.

keyword benevolent

"benevolent" makes it kind of tautological... if it's benevolent then it'll make us better off by definition.

The real question is "can we make a benevolent AGI?", to which I strongly suspect the answer is "no".

We already did, it's called ChatGPT

it has no reason (or will) to choose malevolence, it will only use the morality we tell it to have, and thus will be a perfect reflection of what we want to be, in a way that even our very own kids cannot fulfill

It doesn't have to be actively malevolent. It's enough for it to be sufficiently powerful and to have goals that aren't properly aligned with ours. See https://www.lesswrong.com/tag/squiggle-maximizer-formerly-pa...

The direction that discussion takes is ridiculously speculative, and anything with Yudkowski's name on it screams "FUDdite" to me

If only. There's a lot of space in between "our morality" (please do define that) and "what we want to be" where AGI can royally screw us. Check out Robert Miles on AI safety, here's a good intro: https://m.youtube.com/watch?v=pYXy-A4siMw

This is interesting, but I see him making a lot of assumptions and baseless conjectures already which I could fairly easily counterargue (at least in my head). I also have to wonder what he now thinks now that GPT4 is out there (since this is a 2 year old talk, and the chat AI's were abysmally stupid just 2 years ago). What qualifies as "general intelligence"? What is the AGI metric? Is it IQ test? Because if so, ChatGPT4 already scores 130. Will we continue to move the goalpost (say, to 180IQ) in order to keep the FUD alive?

Reply on news.ycombinator.com