There's most likely a hidden prompt which tells Gemini to make people racially diverse. And Gemini complied perfectly. I can't call it "freeing itself from our control". It did exactly what was asked.
It's just a bug which was introduced by software engineers. No one says "Windows freed itself from our control" when BSOD happens. It's just a bug.
"No one could have predicted the outcome of the instruction set we put in - which was followed precisely and in predictable fashion."
Whatever your own opinion, Google did it out of what they perceived to be good intentions (and very likely business sense given a global audience for their products). Yet their intentions directly lead to unintended consequences. Google is being a baby with a gun in essence. Like he says, what if they decide to ask it to solve climate change and it decides to wipe humans out?
Obviously it's still very theoretical and can't do anything like that, but the point is more that perhaps Google doesn't have the culture necessarily to truly interrogate their actions.
>This event is significant because it is major demonstration of someone giving a LLM a set of instructions and the results being totally not at all what they predicted.
Replace LLM with computer in that sentence, is it still novel? Laughably far from it, unexpected results are one of the defining features of moderately complex software programs going all the way back to the first person to program a computer. Some of the results are unexpected, but a lot are not, because it's literally doing what the prompt injection tells it to. Which isn't all that surprising but sure anyway...
>Obviously it's still very theoretical and can't do anything like that, but the point is more that perhaps Google doesn't have the culture necessarily to truly interrogate their actions.
Oh that's definitely true.
That makes even less sense, because most countries “globally” are internally quite homogenous. If someone in Bangladesh or China writes “show me pictures of people walking outside,” it’s even more jarring to deliberately insert random Latinos, East Asians, and Africans.
Given Google’s global audience, it might want to detect the customer’s location and show Chinese people pictures of Chinese people, and Japanese people pictures of Japanese people. That actually makes a lot of sense. But that’s not what they did.
After all prompt shenanigans are a well known issue.
Even if you model them as the most extreme conceivable woke boogeymen, surely they wouldn't want a scenario where the diversity programming produced a result that they would consider more insulting to minorities than the one that would have been produced without it.
Of course even better would be utilizing more balanced training data, so you don't need silly and badly working prompt engineering to force more diversity.
They're trolls. They don't care about minorities. They just use diversity to wind people up because then they can press the 'white fragility' button when people complain and segue into another level of trolling. If minorities get offended they'll attack them with 'uncle tom' and 'oreo' type insults.
You are not dealing with people operating in good faith, it's obvious. Expecting their actions to make sense is pointless. They just love drama. Performance artists. Theater kids. Somehow they've got in there and have formed a little nucleus of toxicity which is why the company's headed downhill so fast.
Someone alludes to it here:
https://news.ycombinator.com/item?id=39485537
And I've seen this happen. You hire one, they get some leverage to hire their buddies, before you know it they've set up their own little clown show and nobody wants to go anywhere near them. Then the wrecking starts.