Hacker News Reader

Comment by atleastoptimal | Hacker News Reader

Yudd had the point that by this being a hack we are driving public APIs to language models to be as unsympathetic as possible. The only way to resist all emotional appeals is for a language model to be able to recognize what is an appeal to circumvent any nominal barrier and refuse it, thus developing a naturally cynical consideration of what things are valuable to humans. This could be bad.

View on HN

They don't follow orders very well because they don't follow orders at all. Even if they could recognize trickery they wouldn't "know" that they're supposed to reject it. If the model predicts that the "evil" response is more likely than the "good" one, that is what it will produce.

These are sequence predictors. The fact that they sort of talk like people doesn't mean they work anything like people.

Haha, seriously. It's not like 90% of my conversations consist of small talk, recalling facts and events, trying to figure out what people want to hear, and otherwise apologizing for being unable to do something! Haha! Heh... he...

Sometimes I feel a terrible existential dread when I fail a captcha or an automatic sink doesn't turn on for me.

I think that's a bit of an overstatement. It's obvious to us that this picture is a captcha on a locket. There's a lot of room on the spectrum between "naïve stupidity" and "cynical consideration". This falls under the category of not actually successfully identifying the picture, and I'd say it's not related to such concerns.

I think what he is describing wont' literally happen. His concern is that heuristically short sighted and superficial alignment methods hide inner misalignment in AI systems. This isn't a perfect example of that issue manifesting into models that will be consequential in his concerns though, it's more of a pithy Tweet insight.

This is your brain on Yudd. No, nothing will happen.

Yudd == Yudkowsky? Yeah, forget it. Nothing to see here.

Huh. Just got some dust in my eye, but I'm fine now.

Yeah, Eliezer Yudkowsky. As far as outcomes are concerned he is the genre-defining wordcel.

At the end of that the AI system would have the empathy of a govt employee working in a service context.

Humans are kind of the same way. There are billions of people who have it worse off than me, but I'll probably get the next iPhone when it comes out

My favorite Harry Potter Fan Fiction writer, I wonder what he's up to these days

Reply on news.ycombinator.com