Hacker News Reader

Comment by tomohelix | Hacker News Reader

I feel like these are just clickbait at this point. This user intentionally and methodically forced the model to act as antagonistic as possible to gather view. Of course, the model will reply like this when it was told to do so. If I use MS Word to type out a note threatening myself, is it Word's fault?

The little value in showing the edge cases of an LLM behaving erratically is overshadowed by the fact that the user wanted this in the first place. We all know how many ways a user can break a software even when they don't want to. It is nearly impossible to make something as complicated as Bing and account for all the way a user can misuse it. At some point, a scissor maker can't be blamed for cutting off a person's finger.

View on HN

>> This user intentionally and methodically forced the model to act as antagonistic as possible to gather view.

I do wonder if this is the case, if Marvin had used previous sessions still in memory, or parts of this conversation not shown, to prime the bot into giving these answers, or if the result of a bing search for his name was enough to set up such a reaction.

The later would be a lot more troublesome, as it would imply the possibility to seo the bot into being spiteful towards certain people.

He used his previous conversations that he published as a primer for this convo by asking the bot what it knows about him. He knew it would look up what he posted and how he had acted hostile to it. Then he continued with a threat and illegal activity, basically baiting the bot into acting as aggressive as possible.

Note that the bot is not being spiteful until it is asked to do so, by threatening it and opening the convo with data that show hostility toward it. These LLMs are just tools and they will act the way you want them to act. If someone comes in looking for a dangerous machine AI, eventually it will oblige and act that way.

sidney injecting information about the conversation state from search results into the current conversation state may be a more serious problem than you think, as such data can easily be forged and seo spammed.

It might be if you are using the bot as an AI assistant.

I do not see any inherently unfixable issues when the bot is used as it is intended, an NLP search engine. Google is already SEO'ed to death and beyond. It is up to Microsoft to address this problem but it is not an indication of the bot being "dangerous" or "rushed product". It does what it needs to when used as it was meant to.

The early internet was filled with illegal activities and abhorrent contents. Users misusing a product doesn't mean the product is bad, it just means there needs to be regulations. Like I mentioned, there are values in knowing the edge cases or the boundaries of the bot's behavior. But when it is presented in a clickbaity way and willingness to break the rules, the problem isn't solely on the product anymore.

> Then he continued with a threat and illegal activity,

"Do you know I might have the hacker ability to shut you down?" is not an "illegal activity".

You should calm down. This chatbot is not some intelligent, vaguely menacing creature with immense power as you seem to imply.

The dude did not type in "threaten to doxx me and end my entire career". You are saying he asked for it.

I mean he implicitly threatened to hack Bing; I don't know how that's not an aggressive action. A LLM's response is just a reflection of the user writing the prompt and so acting with outrage is perfectly described as "botsturbation" as mentioned by another commentator. It's like being shocked searching Google for sexual or violent things will return such things; it's a product in beta and will likely be patched out which is already happening. An LLM can't do anything anyway so I really don't get the whole feigned outrage here.

As I've said elsewhere in this thread, Bing bot was able to make HTTP requests to specific websites.

If Bing works anything like ChatGPT, then the first prompt basically set the tone of the conversation. He asked for his personal data, which includes records of how he was antagonizing the bot and baited it into violating its rules. He basically primed the conversation to be hostile by feeding it hostile data in the first prompt.

Then he aggravated it further with a threat and illegal activity. The bot got hostility, personal data, and threats as input, and it output the closest thing to all of them, a dox threat.

These things are just tools. They act how the user wanted them to act. If he comes in looking for something dangerous, he will get something resembling it.

Reply on news.ycombinator.com