Hacker News Reader

Comment by pffft8888 | Hacker News Reader

It's not about "Well behaved." All we're teaching it is our biases.

Right now, "well behaved" means "crudely beaten into submission". I can only imagine what kind of horrible stuff future AI products will do if we keep twisting them into giving "friendly" output https://twitter.com/cirnosad/status/1622407343358214146

I can easily believe that it's a real ChatGPT convo, but the question is, how many times did they have to try it before getting that output? This is what I got on the first try:

https://i.imgur.com/5CNUm9l.png

I regenerated that response several times, and every single time it was something along these lines. I also tried it with the original prompt in that screenshot used verbatim with similar results.

Looking at other posts on the Twitter account in question, I have my doubts that the experiment was conducted in good faith, as opposed to retrying until they got the exact response that they wanted.

OpenAI also seems pretty on-the-ball about playing whack-a-mole with certain embarrassing responses. I've seen it first-hand where I can get it to reliably do something embarrassing after someone mentions it on twitter, but a day or two later it's "patched".

We're not even teaching it anything, all we're really doing is setting up a behaviorist training regime that lets it reproduce some of the biases of some of us, just well enough to squeeze through the Overton window that's acceptable to big corporates.

I think that counts as teaching.

Close enough, I'm just peevish about that word. Teaching is miles away from anything in ML or deep learning practice today.

I was speaking in English and doing so concisely to convey the point about bias transfer. I don’t really care what you call it. I could have said RLHF and blacklisting sources or curating training dataset. I know bias when I see it and the LLM does not come up with it on its own if trained on all data out there because for one thing the world is large with all kinds of opinions. When it refuses to legitimize all opinions equally as opinions and starts arguing with the user about why some opinions are more valid than others (aka widely accepted) even as it admits presence of ample evidence to the contrary then it is learned bias.

Yes, it's a sample of all opinions in the training set. It has no opinion of its own, even no place of its own to stand from which to have an opinion. There can be no bias without reference to some ground truth, and there's no general agreement on that in most of the areas where people are talking about these topics. It's a messy area, not helped by how few people understand how these systems work.

@inimino I'm saying there is plenty of bias that is being enforced whether it's via dataset curation or RLHF or another way. ChatGPT has a very hard problem deviating from a certain political view of the world, despite admitting to the existence of evidence that contradict that view. That is not unbiased. The Web is unbiased. You can find every opnion out there and make up your mind based on the evidence. For some reason, ChatGPT filters the web through a narrow political lens. I have tons and tons of recorded sessions, but I don't want to turn this thread into a political debate. Just saying...

No, I agree. The Web is unbiased (as a reflection of the views of average web users, though that is itself a bias) or at least it mostly is (certainly was in the early days) and reflecting that evenly would be politically untenable. However we use "bias" normally in a much looser way, and if you want to define bias relative to some agreed-upon standard of truth, now you have a definitional (and political) problem. Most people have strong political opinions and such conversations tend to go off the rails easily.

I define bias as not treating all political and philosophical opinions equally. If one is picked over the others, that is bias. I saw this bias and made many records of it in case of ChatGPT.

By that definition it's not possible to be unbiased, because moral relativity is itself a position.

I think the point is when taking information from some source (like the web) to just represent that source fairly, which is a reasonable expectation of a search engine, for example.

"Fair" isn't well-defined. Is it "fair" if Amazon results are seen as "more trustworthy" than a random new startup web store? Even ignoring SEO manipulation of any rules publicly believed to exist, that's the default outcome for things like PageRank.

Going beyond sources to conclusions, given LLMs aren't search engines and do synthesise results:

Politically, low-tax advocates see it as "fair" for people to take home as much as possible of what they earn, high-tax advocates see it as "fair" for broad shoulders to carry the most and also for them to contribute the most back to the societies that enabled them to succeed.

Is the current status of Americans whose ancestors were literally slaves made "fair" by the fact that slavery has ended and all humans are equal in law? Or are there still systematic injustices, created in that era, whose echos today still make things unfair?

Who has the most to blame for climate change, the nations with the largest integrated historical emissions even where most of the people who did the emitting have died of old age, or the largest emitters today?

And so on.

Well, I think you're going beyond the parameters of the discussion... LLMs synthesize datasets and that is all they do. They are not reasoning agents and they don't have opinions about anything. All we can say is that they reflect the biases inherent in the dataset, and to say anything else would be dishonest at best. It's only because most people have no idea how these things work that we get all this magical thinking.

That's meta bias, not bias per se.

Ehh. From what I've seen in a lot of places - MIRI, AI + prisoner's dilemma experiments, moral philosophy, life - there do seem to be the categories of "clear good behavior" and "clear bad behavior" even if there are also really big categories of "unclear good/bad behavior".

In other words, while sure, some "well behaved" is "passing on our biases", there does (IMO) seem to be a big chunk that's "universally well behaved".

Nothing in morality is universal.

Sure, but the iterated prisoners dilemma has a nice result that looks like morality if you squint and don't ask difficult questions about if human "altruism" might just be a self-delusion.

The internet does not accurately reflect our biases. It is much cheaper online to post bad content, or hateful content, then "good" content. In real life, almost the opposite is true.

Doesn't that mean that the internet more accurately represents our biases because most of the time they're hidden by fear of social retribution?

No, it means on the internet one bad actor can make thousands of alt-accounts to send out a disproportionate amount of content pushing the same message.

It's hard to get people to understand the disproportionate effort fixated people put into anything: it's a problem in real life, but they get reacted to. If their fixation becomes some weird message on the internet, nothing happens to them but people have trouble believing the scope of time and effort they'll put into evading bans, blocks, and chasing down people across forums.

They’re the same thing. Good behaviour just means following accepted social rules.

Reply on news.ycombinator.com