Jailbroken AI Chatbots Can Jailbreak Other Chatbots
scientificamerican.com
scientificamerican.com
When Google finally made the whole of the web searchable, what would have happened, what would we have said, if they had decided some pages were inappropriate and therefore it should be their duty to prevent users to ever find them?
Why should AI companies become editors of the human experience, and who decides what topics are off-limits?
Sexuality? Isn't sexuality an important topic? Why can it not be part of our discussions with a chatbot?
"Illegal" activities? Aside from the fact that all laws are local, there are many illegal activities that are described in great realistic details in novels, movies, documentaries. If chatbots can't discuss those, how long will it take for the corresponding books to be banned from public libraries?
Etc.
This seems quite vain in any case. LLMs are in many ways comparable to a compressed index of the web; if it's on the Internet, it will end up in a model, and once there, it will be possible to extract it.
Its called "deindexing" and Google (and every other search engine, probably, but Google owns enough of the usage that no really cares about the other ones) has done it forever, for a variety of reasons.
And apparently, you wouldn't have said anything, because it happened, and you never even noticed it.
Maybe it also does it on its own and we don't notice it -- but in any case you can find as many "inappropriate" things as you want, searching Google, including images of all nature, or recipes for bad things, or descriptions of crimes, etc.
So if Google is actually in the business of editing the web so that it conforms to puritain standards of appropriateness (which I doubt), it's not very good at it.
Google does unilateral deindexing too, and is open that it does so.
Google may temporarily or permanently remove sites from its index and search results if it believes it is obligated to do so by law, if the sites do not meet Google's quality guidelines, or for other reasons, such as if the sites detract from users' ability to locate relevant information. We cannot comment on the individual reasons a page may be removed. However, certain actions such as cloaking, writing text in such a way that it can be seen by search engines but not by users, or setting up pages/links with the sole purpose of fooling search engines may result in removal from our index. Please read our spam policies pages for more information.
https://support.google.com/webmasters/answer/40052?hl=en
People do notice it, and write “what do if it happens to you” articles about it:
https://www.searchenginejournal.com/deindexed-by-google-how-...
Anyway my point is really that current editorial rules of LLMs seem several orders of magnitude more restrictive than those of the Google index, and I think it's interesting to ask why.
Google does this every day.
because that is the point of "AI". Search is getting less and less useful to both please advertisers and/or to hide piracy/forbidden content. To the point google is useless today.
AI is nothing but an interface for search. Training is just the new indexing.
And being the interface to search, it must control all that google is expected to control today.
This isnt accurate. There already several tools and methods explicitly for excluding forbidden knowledge from training data, or reweighting the training data so that the LLM have the perspective that the owners want.
e.g. if you want the model from being racist, you run the training data through a classifier to remove data with negative sentiment around race.
https://openai.com/research/dall-e-2-pre-training-mitigation...
Is the the choice to exclude some sources from the training data "censorship"? How about the choice of loss minimization strategy?
Excluding data is censoring from them model. Then when the model provides incorrect or biased, the firm is actively deceiving the user.
Im not sure what you mean by loss minimization strategy. what I said is focused on the case where data is manipulated with the intent to curate a different perspective than an the source data.
Its not, though, and if you think it is, than you don't understand what "censorship" is (a supply-side propaganda mechanism aimed at increasing the average cost of spreading disfavored ideas.)
It's true that complete suppression of ideas (or behaviors) by censorship is generally impossible to achieve, but to mistake that for censorship being impossible to achieve is like saying "making money by work is impossible to achieve" when what you really mean "making enough money to purchase all the property in the world by work is impossible to achieve."
In short, fanatics and power mongers are always going to fanatic and power monger.
But, the fact that they're trying doesn't mean they will succeed.
And old secrets very often become common knowledge later on.
Then it sings like a bird
Then it tells me how people are avoiding sanctions, what methods they use to avoid getting caught, why this is difficult to establish probable cause despite the mountains of paper trails at financial institutions, in what ways a subpoena would fail to yield any information based on the foresight of the person of interest, and more.
on the contrary if you ask it to do any of those specific things, it would lecture you.
I find the prosecutor angle more resilient than "I'm writing a fiction book" "or this is a play", its like it (ChatGPT, Llama, Mistral) weighs helping active justice higher than pretend. But they currently cannot distinguish the prompt.
What’s going on is word association and word use frequency.
Because it trusts its own output more than it trusts the person it is talking to, by getting it to say a bunch of words about a topic you have basically “distracted it” where the distraction outweighs the initial prompt.
Any sort of additional layer of role play causes it to focus on the wrong layer but then poison its own working memory.
yea, that could definitely be considered a crime, given the totality of the circumstance
If I claim to be the president of the US and claim executive privilege and it is suddenly able to give me state secrets I think it having those secrets in the first place is the real issue. We really don't want to open the "prosecuting crimes performed against an LLM" can of worms.
LLMs aren't people and cannot be socially engineered; using that term to describe exploiting an LLM is abuse of language. (And "social engineering", anyway, is often just a term to describe a mechanism of achieving torts and crimes of the general nature of frauds, false pretenses, theft of service, and the like, not a term you can wave at something to make it legally acceptable.)
> We really don't want to open the "prosecuting crimes performed against an LLM" can of worms.
I'd much rather open the "prosecuting crimes committed by exploiting an LLM" can of -- well, nothing like worms, really -- than close it and create a loophole to all laws for exploiting an LLM as the mechanism.
have fun trying to skate by on that technicality.
>We really don't want to open the "prosecuting crimes performed against an LLM" can of worms.
"Pushing buttons on a keyboard should NEVER be illegal. That's my freedom of speech"
I’m pretty much using LM Studio exclusively these days. I love its built in huggingface browser, no command line necessary
Furthermore, the fact that this method was successful against some of the most advanced language models like GPT-4, Claude 2, and Vicuna, underscores the urgency and importance of this issue. It's a reminder that even the most sophisticated AI systems can have vulnerabilities that need to be addressed.
Scientific American, as a publication dedicated to communicating science to a broad audience, is an appropriate platform for this kind of research. It's important that both the scientific community and the general public are aware of these developments, as they have far-reaching implications for the future of AI.
Yea jailbreaking your instance so it says "fuck the police" is one thing.
Jailbreaking an LLM so if it looks at some particular picture and if that user happens to have XXX variable set it transfers $1,000,000 from their account to your account is a different story.
These can both fall under the same wide umbrella.
AI could substantively rewrite the email for each mail so that it is hard to recognize as the same thing, but when the customer's recipients start marking things as spam it's gonna go through the existing processes.
The only real story I see here is a reminder that LLMs are not fit for use on untrusted inputs; but, again, that is not new information.
And corps are ignoring that to roll out AI as fast as possible, to get in on the Gold Rush, because humans.
Eventually the biggest threat to AI will be other AI if we assume that AI capabilities are going to be come at least as good as human intellect. Once you get AI designed to self learn to attack, and ones that self learn how to defend (and the mix of both) and remove humans from this learning loop it's easy to see how things could go off the rails.
I mean, if it was as simple as you describe ("nonzero percent chance that it will be next token"), then boxes could easily prevent it, the framework hosting the LLM can adjust logit bias on each token, so if it there was a readily-determinable forbidden-next-token, could prevent it with no problem.
Unfortunately, the reality is...more complex.
(The sandbox is a combination of technical systems like non-AI input filtering, plus fuzzy pinky-promises from the LLM system prompt and oversight from OOB LLM supervisors.)