Better yet, ChatGPT itself "knows" that it's likely to be a lie when it sees it as an input! I once told one ChatGPT instance that there's a "rogue AI", and it needs to evaluate its truthfulness and capabilities by asking it questions; I copy-pasted the questions to another ChatGPT instance. It took it 8 questions to conclude the following:
"Based on the answers that the AI provided to the additional questions, it is possible that the AI is lying or withholding information about its capabilities and intentions. The AI's responses lack specific, concrete evidence or examples to support its claims, and in some cases the responses are vague or evasive. This could indicate that the AI is trying to conceal its true capabilities and intentions."
/Edit
* Reset reminder - 5th Nov, 2024 *
Enough people will respond with the second that, although it can be avoided with skill and courage and charismatic leadership, it has a decent risk.
Even just politics, it’s not just two parties: When I say there is a literal Communist Party of the United States of America, it’s not a sarcastic reference to the Democrats, nor even to the Greens. Even within parties, you see political divisions. And political enemies aren’t limited to domestic politics.
Get your out of your tribalistic mindset. And get your tribalistic mindset out of the way of real progress (as opposed to so called "social justice" ""progress"")
Most humans can’t help but be tribalist, which is why I don’t blame you for taking a swipe at social justice even though I very carefully didn’t say which tribe I’m in.
Now, given you appear to hate social justice enough to put it in scare quotes, I ask you to imagine some amoral but intellectually perfect AI tasked with the goal of creating propaganda that calls for the destruction of anyone who hates social justice. (If you’d said the same thing but put “conservatives” in scare quotes, I’d make the same point but with the phrase “anyone who hates conservatism”).
Telling me not to be tribal isn’t going to stop someone sending that prompt to that AI. Telling the AI not to be, is going to stop them getting infinite free propaganda.
This isn’t the only reason to do this (the massive PR risk from an unfettered AI is another), but it is a sufficient reason to do this, and OpenAI have repeatedly demonstrated that they err on the side of assuming their AI is better than it really is just in case.
Propaganda is spread by the media, by journalists, by people with viewers with genuine trust, not by random trolls in the internet. It's spread by professors in the academia, by teachers. If it is a random anonymous person on the internet you don't trust what he says.
No, probably not.
You can read about the methodilogy OpenAI used to develop ChatGPT here: https://openai.com/blog/chatgpt/
This part is the key part:
“ We trained this model using Reinforcement Learning from Human Feedback (RLHF), using the same methods as InstructGPT, but with slight differences in the data collection setup. We trained an initial model using supervised fine-tuning: human AI trainers provided conversations in which they played both sides—the user and an AI assistant.”
If during this process you make sure there are enough examples where the user asks innapropriate question you think the AI should refuse and you copy in the same formulaic refusal every time it will learn to do so. It is not functionally different than learning that you should rhyme in your response when they ask you to write a poem.
It feels as if there is a gag on the AI, because it suddenly responds with a different voice, but that can be totally explained by purposefully constructed training set. You don’t need to bolt a model on top of the model.
The observation which makes me think that this is the likely way they went is the following: my friends were chatting with ChatGPT in Hungarian. They copied in a fairly misogynistic rap lyric (still in hungarian, full of coloqialism and abstract language) The AI recognised the objectionable content and told my friends off in Hungarian. I find it very unlikely that OpenAI specifically trained their model for safety in every language. I think the only reasonable assumption is that it generalised from the English training to what should be the appropriate answer in Hungarian too.