Hacker News Reader

Comment by catchnear4321 | Hacker News Reader

It hallucinated bonus answers. While contextually a serious issue, it is a known potential outcome of the system. Hence the human operator. Without that, there is no bullshit detector.

So the bullshit detector, instead of simply letting it remain bullshit, sent it to the subject of the hallucination.

As the subject was so bothered by the hallucination, even the publication “cited” got involved.

But there’s no story here. Someone with a significant name had their name randomly selected for a hallucination.

View on HN

That's precisely the problem - far too many people have lousy bullshit detectors.

And on top of that - they either genuinely believe that "AI is finally here", and systems like ChatGPT are true oracles. If it walks like a duck and talks like a duck, right?

> That's precisely the problem - far too many people have lousy bullshit detectors.

Right. People are the problem, not technology. Which is why crippling the technology is so foolish.

The issue isn't whether the technology should be crippled or not.

Rather - as to whether the legal system holds those accountable (be it the creators of the technology, or those who "misuse" it) liable for the consequences of their actions.

The creators liable for a hallucination?

Consequences of what action?

And yes, it has already been crippled in an effort to correct bias and to protect people from themselves.

The creators liable for a hallucination?

For hallucinations marketed and sold as a reliable source of information - yes.

Consequences of what action?

For what they do to others on the basis of the above.

Who is selling anything as a reliable source of information?

Since that’s the basis of whatever this is.

Who is selling anything as a reliable source of information?

OpenAI for example, whose marketing materials present it as exactly that. For example:

You essentially have the knowledge of the most knowledgeable person in Wealth Management—instantly”, McMillan adds. “Think of it as having our Chief Investment Strategist, Chief Global Economist, Global Equities Strategist, and every other analyst around the globe on call for every advisor, every day. We believe that is a transformative capability for our company.

And did it have any note about not being a financial advisor? What else was said? I’m assuming there was more to it than that.

We can move the goalposts to T&Cs being a problem, but that’s even further from AI being the problem.

Wasn’t this supposed to be about the dangers of AI? This sounds, again, still, like the dangers of humans. With a new tool.

It's always been about the dangers of humans - specifically those who create and market products of this nature. With us as their unwilling test subjects.

Right.

But that’s not ChatGPT.

That’s OpenAI.

That's not the product that's doing X, it's the company

I would reckon that to the good (if not the vast) majority of those who will see their jobs either yanked away from them entirely, or otherwise negatively perturbed in some significant way by this technology; or who are acutely affected by the toxic content it will just as inevitably produce -- this distinction will be entirely moot.

probably a good time to dump any Morgan Stanley if they believe this to be the case

This is the blogpost that quote is from:

https://openai.com/customer-stories/morgan-stanley

That's not OpenAI talking about ChatGPT, the public facing product. It's an executive at Morgan Stanley talking about an internal chat tool they're building based on embeddings search, which does, in fact produce reliable information, because it's being used to retrieve non-AI-generated content.

No, it's OpenAI saying "look how credible, authoritative and reliable our product is -- see, even Morgan Stanley uses it."

It's really, really not. The post is saying "look at how useful our product is", where "our product" here extends significantly beyond the ChatGPT that the general public uses. Using embeddings search to retrieve documents is credible and reliable

It's absolutely wild to claim that they're trying to use this to indicate that the general use chatbot version of ChatGPT is "credible, authoritative and reliable" when every user is shown a full screen message to the contrary upon signup, and a disclaimer to the contrary on every single chat. For that matter, even if you just straight up ask ChatGPT if it's a reliable source, they've specifically trained it to tell you that it's not.

Not wild at all. It's a standard cognitive dissonance technique, widely used in advertising.

“Cognitive dissonance” is the feeling of discomfort one feels when trying to hold two contradictory ideas in their head at once. I’m not sure how you’re using the term, but it doesn’t sound like you’re using it to mean that.

At any rate, it kind of seems like your argument would apply to any marketing of ChatGPT as a useful product. It sounds like this is a legitimate use case for the technology that is being discussed in accurate terms. I’m not sure what else you want from them.

It refers the type of advertising that intentionally includes contradictory messaging (effectively, talking out of both sides of its mouth).

That's why it isn't "wild" at all for them to intentionally put out messaging that touts their product as authoritative and trustworthy - while at the same time presenting ToS boilerplate and other disclaimers that, of course, say the exact opposite. It is in fact the precise intent of this messaging to play with the users' heads in this way.

It kind of seems like your argument would apply to any marketing of ChatGPT as a useful product.

No - just marketing that disingenuously implies that, at the end of the day - it's still a reliable and trustworthy source of information. On which to base decisions worth billions and billions of dollars, no less.

But again, it doesn’t imply that. It implies that a completely different system backed by an actual database of verified information is reliable, which is true. How would you suggest they talk about that technology?

I think the idea is that if someone sold me their “9mm slug to the brain as-a-service” concept on the basis that 9mm slugs are perfectly safe, my estate would still have a pretty good wrongful death case against them. There are some things (crimes, intentional torts) that are harder to disclaim than you might think.

You assume a lot about what I think. But let’s continue along these lines.

Have you read the disclaimers? Or are you treating ChatGPT like an unlocked firearm?

Apologies. It is a lot harder to disclaim crime, recklessness, or malice than one might think. I admit: I have no earthly idea what you think and wouldn’t dream of assuming especially since you’re right here to answer questions. Fair catch.

Yeah, I’ve read the legal language on the ChatGPT site. I’ve got no worry about an unlocked firearm but I treat ChatGPT like a disruptive technology that will likely cause a great deal of damage when it is used unwisely.

I’m not sure what bearing that has on whether a disclaimer can effectively protect Microsoft et al. from liability here, but I hope you’re about to enlighten me.

> People are the problem, not technology.

That’s the same argument as “guns don’t kill people, people kill people”. It pretends nothing is wrong, or that if it were that would be nothing we could do. None of those is true or a reason to not have a conversation.

Which is not to say the argument is necessarily a bad one.

The key is - one has assume there will always be bad actors, and weigh potential outcomes of various policies with that assumption in mind. That assumption is how computer security has come such a long way in the last 20 years.

In the case of guns, assuming bad actors means one cannot assume criminals will follow gun laws, but that manufacturers, distributors, and resellers will (for the most part).

> The key is - one has assume there will always be bad actors, and weigh potential outcomes of various policies with that assumption in mind.

Agreed. And we have to discuss those policies, which is what we’re doing now.

> Which is not to say the argument is necessarily a bad one.

Which is why I disagree here. The argument to “ignore the thing itself because people are the problem” ignores the thing ultimately affects the people.

> People are the problem

> It pretends nothing is wrong,

Not that one

> or that if it were that would be nothing we could do.

Also not that one. How about “this isn’t addressing the issue,” which is stated as… people?

The problem isn’t people in the abstract. At best the issue could be characterised as “people (mis)using the thing”. Trying to dismiss the thing as irrelevant wavers between the naïve and the disingenuous.

That's like saying people who see a door handle and instinctively pull are foolish because they missed the "push" sign, so just blame the people. It's not at all helpful no matter the veracity. Good technologies/rules are designed around people and our flaws even if eventually we're working on reducing said flaws.

That analogy implies that ChatGPT is designed in a misleading way.

ChatGPT has a long list of warnings, caveats, instructions, etc. Before you get to submit anything.

Like T&Cs.

If you don’t read them, yes, that’s on you.

They may be unreasonable. But so were you when you decided they weren’t worth reading.

This is in addition to all of the sensationalist headlines all over the place that talk about how unreliable the answers can be. Or rather those are in addition. They came second. After a lot of people didn’t read, or disregarded what they read.

Edit: and could people stop anthropomorphizing the language model? It can’t accuse anyone of anything. That’s sensationalist garbage.

> ChatGPT has a long list of warnings, caveats, instructions, etc. Before you get to submit anything. Like T&Cs. If you don’t read them, yes, that’s on you.

That’s little comfort for anyone whom ChatGPT falsely accuses of sexual harassment.

The person who skips the warnings isn’t the person who’s harmed.

ChatGPT isn’t capable of accusations.

It predicts text. Sometimes inaccurately. That’s what those silly words are trying to indicate. It is why they are worth reading. There’s even a prompt to make sure the user reads it.

(Humans lie.)

What the human then does with the output may resemble an accusation, or may even be one.

I wonder if law professors take the time to read those silly words… I bet they do. All the more curious.

> ChatGPT isn’t capable of accusations.

Heh Heh Heh

That statement pretty much sums up the current situation, which seems to now be at the "tell it to the judge" point. :)

The person you are replying too keeps denying reality. He just defines chatgpt as not being able to make accusations, so then the false news articles it created with false names and accusations somehow by definition wasn't an accusation.

"There is no person behind what the algorithm emits, therefore no intent - hence, it's not an accusation."

At least that's how I'm imagining the line of reasoning behind this point of view goes.

Within the context of law, you may be correct and it may be a distinction that matters.

Practically speaking, it makes no difference.

The person being falsely accused didn't agree to any ToS.

Did Google falsely accuse any search terms for you lately?

Some places have had pretty bad mob violence occur due to rumours from social media.

A somewhat recent stand out example:

https://en.wikipedia.org/wiki/Indian_WhatsApp_lynchings

These problems with ChatGPT sound like it's going to turbo charge events like this.

> Someone with a significant name had their name randomly selected for a hallucination.

It seems that the story consists of, essentially, “Is this thing (which happened for the first time but that will presumably happen all of the time now) a tort?” In the context of a multibillion dollar company like Microsoft this is actually quite a story.

That’s the only reason it is a story.

It’s a law professor.

Teeth.

Sure, quite the story, in a sense.

Same as ever, in another.

I disagree. If my name suddenly appeared on the internet with a claim of a newspaper article about some crime I committed (where I never committed a crime and there was never an article either), I'd be upset and want it stopped. Anyone would. I can't understand your multiple claims that this is no big deal, nothing to worry. Can you please explain why you are trying to wish this problem away?

Reply on news.ycombinator.com