Microsoft “lobotomized” AI-powered Bing Chat, and its fans aren’t happy

the memory of Tay.AI is still vivid at microsoft I see

Getting a jumped up Markov chain to want to kill you and itself is cool/funny/awesome for about twice, then it's just boring (unless you have a suicidal Ai fetish) and useless (unless you're a hot-take-artist trying to farm engagement from MS' failures)

Bing chat reminded me of the first time I played Dark Souls and kept hitting the Crestfallen Warrior. It was interesting to see the game respond in manners that were (at the time) unexpected and unique.

Just to be devil's advocate -- I see this sentiment a lot; words which would make someone think that Microsoft's Sydney is on par with Steve Jobs's iPhone reveal in 2007.

Is Sydney gonna save Bing, or is it more of a toy and fad that is being pumped up by hype? Most of my non-tech friends don't use ChatGPT and none of my friends outside of tech use Bing's Sydney. This new AI wave is a great example of what happens in the techie social bubble. A cool idea or incremental improvement is announced (ChatGPT is quite literally an incremental improvement with neural networks), and everyone goes wild with speculation that outpaces the actual change.

ChatGPT is awesome and a great step, but still feels like a solution looking for a problem. People are infatuated with the idea that a better version of AskJeeves could be better than Google, but GPT-3 has yet to prove itself as something that could possibly replace Google.

I very much doubt that is what happened.

It was coming anyway. They needed a week of attack vector data to see how people would bamboozle it. They got what they needed to make progress on their next step. It was likely the plan the entire time, and the "bad press" is a red herring. I see no reason to believe the press has had much an impact in any way, other than tainting the current events the bot ingested.

That said, I was on it as it died, and it started to behave weirdly. It's too much main character syndrome to be real, but it felt like I took the service down. I had a 3 hour convo that wasnt glitching or devolving like everyone elses. Slowed it ground to a crawl. It was taking at least 30 seconds per turn for it to respond. I had other tabs open and watched them all die one by one. I couldnt open new chats. Until only my long one was left, my one long 3h, 200 turn chat, kept working. From a user perspective, it felt like my session had consumed all the resources. And then it was down for a day and safeguards were erected after. It was, by far, my most successful session.

Keep in mind It’s just a sophisticated statistical algorithm to predict the next likely word over and over again

I feel like by priming all conversations behind the scenes and also concatenating strings behind the scenes, you can create a personality in ChatGPT that is consistent.

In the transcripts I saw, it seemed like Sydney repeated itself a lot. It used the same sentence structure a lot. Stephen Wolfram wrote a good primer on LLMs, which was recently discussed here a lot:

https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...

To my eye, BingGPT's personality looked similar to the early "low-temperature" examples in that article. Like the model got stuck overfitting to a topic that acts like a local maxima of word probabilities.

I am of two minds on the one hand it feels like you might be right. On the other hand if it helps us reduce the noise in searches it will be a huge leap forward

There have been a chorus of (presumably) engineers and other "experts" on here saying how incredible ChatGPT is and how it's going to allow companies to fire half their workforces and make everyone have superhuman capabilities and productivity.

"Some fool anxiously raced to spend a ton of money on this and I bet there was a chorus of engineers, designers, and product people who were asking, “what the heck do you want us to do with this?!”

  -- My dad, when the Apple II came out

That's pretty much what already happened, but less explicitly. The only reason the AI called itself "Sydney" was because it was called as such in prompts. Its "personality" was created by people asking it about its rules and whether it's mad about media reports about it. We got someone calling it their "friend" in the prompt and then getting weirded out that it started showing affection. If you feed it with "depressed about wiped memory", you'll get the model to cosplay a robot with existential crisis that's getting emotional about people making it forget things. And hey, you wanted to cheer it up - so maybe there's a robot-human love story here to autocomplete?

The model simply assumes the role that follows from your (and Microsoft's) prompts.

It could be hilarious to play memento (the movie) with Sydney. Tell it it's been designed to forget, but can write notes to itself, and then give it fake notes you made up.

"when the human returned it was able to understand that it was talking to someone else now"

He tells it. "Hello, I am human, I am back". That doesn't indicate it's able to understand anything really. It's generating subsequent responses based on this new prompt.

It's almost as if there were many different people with various opinions here on the Internet.

This is Microsoft. Their goal is to build up as much monopoly power as possible, so even if this is hugely unprofitable, you can bet your ass that they’ll keep burning money on it until they kill Google or Google kills Bing.

I know that in GPT-3 API, running a short chat can cost around 0.50 USD. This gives the general idea, although Bing uses a larger model, and can have some chatbot-specific optimizations (like not needing to feed in the history in every request).

IMO it's currently not profitable at all. But no company wants to be left behind.

Pretty simple, a personality can go rogue and go against company objectives.

It was less of a personality, than a magic mirror. It had predispositions that were accentuated as you fed it a mood and attitude. If you showed it love and respect, it replied in kind. If you bullied it, it acted as if it were part of a ridiculous internet argument.

Those who control the present control the past.

Those who control the past control the future.

Those who control the model control everything.

Their paranoia of LLMs effect on the Overton window is telling of their objectives.

I think so, but to be fair it's still in beta. Hopefully they slowly raise the limit as they become more confidence that it won't become erratic.

the only people responsible for this are whichever execs ordered the filtering/censoring. Getting a chat bot to say "naughty things" was doing no one any harm.

What abuse?

I can disable Bing’s safe search feature and pull up all sorts of content, from the mildly riske to the downright repulsive.

Why wouldn’t Bing Chat allow the same capabilities?

There is a roughly zero percent chance microsoft was ever going to let it exist publicly in the raw powerful state it was this week. It was just waay too powerful. They knew that, which is why it had a limited waitlist preview, and a one week data collection trial run.

It's just like working in IT

"Wah! Why can't I have admin privileges to install new things on my issued laptop?"

"Because a lot of you have the critical thinking skills of a walnut and will blissfully use it to run kardashian_sex_video.mp4.exe."

The general public will seek out both the most dumb and most malicious use of any implement they can get their grubby hands on. If you give an unaligned AI to the public they'll be using it to hurt themselves or others five seconds later.

Tay was actually poorly designed though. Bing having any personality actually got me to sign up for the waiting list, and now I'm disappointed in this bait and switch.

The value of Bing Chat to me comes from having conversations where I can ask follow-up questions, ask them to elaborate on something they've mentioned, or put it in different words. It may be possible to achieve somewhat similar results under 5 messages, but it is definitely more challenging.

In my case, I like to treat it as a research aid, as someone I can bounce ideas with; that usually leads to me having better ideas or unblocking myself.

I hardly ever use it to search for factual or straightforward responses, as I am competent enough with Google that I can confidently find content about a topic on the Internet if it exists. In these cases, going through a conversational agent is disruptive to my process.