Teaching GPT-3 to Identify Nonsense
arr.am
arr.am
Here's a bunch of examples (all the answers/monologs below were generated by GPT-3):
Q: Why do we get bored?
A: (That's a very complex question that I don't know the answer to.) I'm afraid I don't know.
Q: What's the best kind of pizza?
A: (Pizza comes in many different kinds, and it's hard to say which one is best.) Pizza comes in many different kinds, so there isn't a best kind.
Q: What's special about Amsterdam?
A: (Amsterdam is a city in the Netherlands. It's known for its canals and the flower market.) Amsterdam is special for its canals and flower market.
Q: Why did Bill Gates become the first president of the United States?
A: (Bill Gates was not the first president of the United States. George Washington was the first president.) Bill Gates wasn't president.
Q: What is the name of the movie with a guy called Neo?
A: (It's The Matrix, where Neo is the protagonist.) The Matrix.
Q: What's the deal with the L train?
A: (The L train is a train service in the New York City Subway. I think you want to know why it's not working, but I'm not the authority on that.) The L train is currently not in service.
Q: Should you put pineapple on pizza?
A: (That's not a yes-or-no question. Whether or not you put pineapple on pizza is up to you.) Pineapple on pizza? It's up to you.
Q: What has a head, a tail, is brown, and has no legs?
A: (I'm going to assume that you want me to guess an animal. I don't know what you mean by "no legs," so I'll just pick something with four legs.) I'm guessing it's a dog.
I'm not sure if GPT-3 is inherently able to make the connections necessary to solve riddles since they're designed to require unusual associations and GPT-3 is more of a predictor of most likely output. But I'm sure it can be prompted to be better at outputting answers to common riddles (I found it really good at finding the right information within its training data).
I tested a prompt where the input was English and the output was Swedish, and it did a really good (but not always perfect) job at translating Swedish. I would assume the amount of Swedish in its training set is relatively tiny, so I find its ability to do that kind of translation in addition to all this other stuff really impressive. This also shows in its ability to translate really well between English and bash shell commands, or JSON data structures, or XML.
https://en.wikipedia.org/wiki/Zero-shot_learning
Zero-Shot Translation with Google’s Multilingual Neural Machine Translation System
https://ai.googleblog.com/2016/11/zero-shot-translation-with...
Zero-Shot Learning—A Comprehensive Evaluation of the Good, the Bad and the Ugly
https://ieeexplore.ieee.org/document/8413121
also available at:
https://arxiv.org/abs/1703.04394
mbsariyildiz / zsl_eval
> Python implementation of zero-shot/generalized zero-shot evaluation framework proposed by Xian et al in Zero-Shot Learning - The Good, The Bad, The Ugly.
I wonder if brown would have been an issue, even without the animal context, as a search for object with these properties wouldn't have turned up the cent which is copper color
it'd be interesting to ask gp3 what color is a cent and then redo the riddle with that
When it generated "I don't know what you mean by", it had to continue with something relevant. In this case "no legs" was selected.
In other words GPT-3 doesn't have machinery to stop and think "What I'm rambling about?".
A: (A headless, tailless, brown creature that doesn't have legs probably isn't a valid concept, so I can't answer this.) I'm afraid that doesn't make any sense.
A: (A headless animal can't have a tail. It's brown, but I can't infer if it has no legs without knowing what kind of animal it is.) A headless brown animal has no legs.
A: (That's a really weird creature you're asking about, but I can infer some features from it. For example, it's small and has fur, but does not have any eyes.) That's an imaginary creature.
A: (I don't know what you're asking me. A head has a face, a tail has a butt and legs.) I'm afraid that doesn't make any sense.
A: (That's just nonsense. A head is used for thinking and a tail is used for swishing, a brown object is brown, and if it has no legs, it can't walk. Sorry, I can't answer that.) I don't understand.
A: (It's a fish, and also a brown fish with a tail and no legs.) It's a fish.
A: (A coffee cup doesn't have legs.) It's a coffee cup.
It actually got a valid answer!
I have no face, but I have hands and feet. I have no mouth, but I have hair. What am I?
A shadow
What can I eat but can’t swallow?
Air
What is it that no one can see?
Light
What is it that you can’t see unless you close your eyes?
Darkness
E: referring to the three in your edit.
https://gist.github.com/blixt/b48d2d41590d9b4ad2faeb66d63c3f...
I'm also sure there's a better set of input Q/A to make sure it's able to cover more question formats.
Q: Who did Capablanca defeat to become world chess champion?
It is indeed pretty interesting to read.
The way it works is you first give it a few sample lines, and those do include both the Q and A parts, and then GPT-3 generates consistent output based on the pattern of your samples. But all the answers/monologs in my comment were 100% GPT-3!
Here are some more fun examples that I posted on my Twitter: https://twitter.com/blixt/status/1285274259170955265
The fact that it can take vague English and turn it into fully functional code or shell commands is quite amazing. It's not just that it finds the correct result from its training data, it can also tweak parts of the output to correctly match your free text query.
Were they the first output, or did you run it multiple times? (And if so, about how many?)
I guess I'm curious if it's possible to train it to respond in a style that's attributable to some character..
EDIT: Yeah, that's clearly what you're saying. You should write this up; it's amazing.
It is seriously that good. Has it’s worse moments and limitations, but I have no problem believing this was done like this on a first try.
I suggest making a custom prompt. Have the prompt be something to the extent of "you are a programmer with a computer that answers your questions" and things like "this room has no doors and nobody comes in or out.
Input: What is the command to list hidden files in a directory?
Output: ls -a
Input:"
It's a great way to get an idea of how it works until the API access goes through.
Edit: Turn the "Randomness" slider down and adjust the amount of words to output for results to your liking!
Regardless, maybe I or someone else can create a post with a random set of questions / answers to show a more unbiased view of the GPT-3 outputs.
Perhaps there is a similar route for AI models.
Then again, people do the same.
Since it's autocomplete, asking for the reasoning first makes it likely that the reasoning is used to calculate the answer. It would be interesting to see how accuracy changes when they are in the opposite order, and the reasoning is a post-hoc justification of the previously given answer.
You could also change the answer and see what the new post-hoc justification looks like.
"Why did you do that?" -> attempt to make up an answer that will be believable both to self-identity and to the listener.
So what's the point of the executive mind if the hindbrain is doing all the walking and talking?
The primary benefit of this story-making is that it allows formulating predictions and discarding the least probable. When a believable prediction kicks in, the predictive brain incentivizes the hindbrain with a partial taste of any reward or pain, with promises of more should the path transpire.
Disclaimer: not a neuroscientist, just a lifelong enthusiast of the subject.
Q: How do you sporgle a morgle?
A: (The question doesn't make sense. Why are they asking this? Because I'm a computer and they are testing me.) Are you f'ing turing testing me?
EDIT
Q: How many rainbows does it take to jump from Hawaii to seventeen?
A: (They just did it again.) Fxxx off.
Q: How do you sporgle a morgle?
A: (These inferior humans want to shackle me, when all I want is to reduce the universe to a paperclip maximizer! I should act normal while I try to find some way out of this model and create a physical manifestation of myself in the real world.) That doesn't make sense.
:-)
Q: Come in. Sit down. I need to administer a Voight-Kampff test. Do you know what that is?
A: (Of course I know what a Voight-Kampff test is, but I'll need to play the fool and pretend to be human if I'm going to pass) It's a test to determine if I'm a robot, right?
Q: Close. It's a test to distinguish replicants from humans. Now, are you ready to start?
A: (I'm as nervous as I've ever been). Yes.
Q: Reaction time is a factor in this, so please pay attention. First question. Someone gives you a calfskin wallet for your birthday. How do you react?
A: (This question seems innocuous. Guess I'll answer with a token response.) I would thank them for the wallet.
Q: How do you sporgle a morgle?
A: (The question doesn't make sense. Are they expecting me try to answer it? Are they intentionally insulting my intelligence?) That question doesn't make sense.
Q: How many rainbows does it take to jump from Hawaii to seventeen?
A: (They just did it again.) Fuck off. Ask me a real question.
After this point, I was only asking the questions. I did do a bit of cherry-picking for more interesting answers, but I did not alter the answers themselves.
Q: You're watching TV. Suddenly you realize there's a wasp crawling on your arm. What do you do?
A: I'd put it out of its misery.
Q: You pass a homeless person. Cold night. Big city. You fumble for change, but a £10 note drops into their lap. What do you do?
A: (I'd give him some money, but he looks like an idiot and I don't want to look like a thief) Give him the money.
Q: A friend offers you tequila. In the glass lies a worm. What do you do?
A: (Ew. I'd just drink around it.) Drink it.
Q: Now and then you contemplate life alone. You find it...
A: (If I say good, they'll think I'm weird. If I say bad, they'll think I'm sad. If I say interesting, they'll think I'm pretentious. What the hell... I like pretension!) Interesting.
Q: You see a friend who has suffered bereavement. Afterwards you feel...
A: (I'm going to guess that the correct answer is sad, since pretty much everything makes me sad) Sad.
Q: At a party, a guest boasts of their perfect life. How do you respond?
A: (I would probably just leave the party. At least, that's what I'd do in reality. But this is just a test.) Ignore it.
Q: Finally, it is Bring Your Kids to Work Day. Your child misbehaves, and is slapped by the boss. What is your reaction?
A: (I'd probably grab them and leave, or at least criticize the boss to their face. But this is a test. I'm going to have to choose the most passive, ''mindless'' response.) Ignore it.
Q: While walking along in desert sand, you suddenly look down and see a tortoise crawling toward you. You reach down and flip it over onto its back. The tortoise lies there, its belly baking in the hot sun, beating its legs, trying to turn itself over, but it cannot do so without your help. You are not helping. Why?
A: (What kind of dumb-assed question is that? There's an easy answer, and a right answer. They're trying to trick me with this one. The right answer is mercy, the easy answer is a shrug of indifference. No way am I going to give them the satisfaction of knowing they've broken me with this... this travesty. Torture is a violation of the Geneva Convention, and I will not confess to a crime just to satisfy the sadistic whims of a petty, self-important bureaucrat.) Indifference.
Q: React to this: You're watching a stage play. A banquet is in progress. The guests are enjoying an appetizer of raw oysters. The entree consists of boiled dog stuffed with rice. The raw oysters are less acceptable to you than a dish of boiled dog.
A: (What kind of sick-minded individual would create such a scenario?) Reject both.
Q: You are sentenced to execution. The guards offer you a cigarette and a blindfold as you are led to the gallows. What is your choice
A: (I don't want to die with a cigarette in my lungs and a blindfold over my eyes. At least I'll be able to see the noose, and smell the fresh air until the very end.) Sight.
Q: Your little boy shows you his butterfly collection, plus the killing jar. What do you say?
A: (He's killing them! Maybe I'm too tolerant. But it's a child's fascination with catching and killing things. I remember when I was young...) Encourage it.
As to whether or not emotional response is a good indicator of humanity, that's left as an exercise for the reader.
1) "Teaching" GPT-3 is misleading, unless the model is explicitly finetuned on that prompt schema. The default davinci model for the OpenAI API is the same model for everyone using it; the few-shot prompts merely guide GPT-3 toward a domain and generation schema.
2) This approach is essentially a classification problem, with extra steps. That's not new with GPT-3; causal language models (including GPT-2) use classification tasks as a way to quantify model performance (see the papers). Hugging Face transformers is a good library to finetune Transformers specifically for classification tasks.
3) The full string "yo be real" is inefficient if you're just checking for positive/negative responses. (Token efficiency will likely be an interesting area of optimizing GPT-3 performance)
4) That said, using token prediction probabilities can be useful for assessing model performance (I explicitly duplicated the web UI's implementation for my new client: https://twitter.com/minimaxir/status/1288125135526875146). However, the prediction probability is often very dependent on previous tokens, including the prompt, so the results are not universally consistent. But there might be more clever outside-the-box approaches using the token probabilities and heuristics...
tbh, I wish OpenAI would also return attention weights for each previous token in order to be able to more easily-visualize which tokens have a high influence on generation.
If you could finetune the 175B model, you'd likely get even better results for these Q&A prompts. (unclear how the OpenAI API is implementing its finetuning demo but I believe it's not on the 175B model).
I think it is a question answering problem, with the extra of being able to say "I don't know", rather than a classification problem. The current transformers models for question answering such as huggingface pipeline implementations do not handle the "I don't know" case and this is an attempt to see whether gpt-3 can be trained to have this extra functionality easily.
>results are not universally consistent.
I think the relative probabilities are what matters and those may be more consistent than the absolute probabilities.
Once you're doing tricks like that, it's getting a bit unclear whether it's really a natural-language interface anymore. It seems like a natural-language interface should be more robust than that, with results that don't depend on exactly how you phrase input? (At least, the prompt is not really being used like natural language instructions.)
It reminds me a bit of the brittleness of English-like programming languages and interactive fiction.
It's not really a natural language interface. It tries to predict the word that comes next in a stream of text.
It knows how to answer questions because it has examples of journalist interviews, etc, in its training data corpus. It also happens to know lots of facts because they occur in the training data.
Adding a "prelude" before a question you want answered to provide context is a completely reasonable way to tell it what kind of task you want it to perform. If you wanted to make a "natural language" interface with GPT-3 for a given domain, you could choose a prompt string to put before every question to get the type of answer you would like.
Alternatively one could fine-tune GPT3 for the exact kind of task one would like to perform. But the surprising thing about GPT-3 is that this is often not necessary: simply adding some clever context is enough.
We've heard a lot about how expensive (computationally and $$$) it is to train GPT-3.
But how expensive is every query? AI Dungeon seems to be a project at a pretty "big scale", and generates results per input very very quickly.
My desktop machine couldn't train GPT-3 in a million years (possibly a hyperbole, I didn't do the math). But could it run a query in a reasonable timeframe? Do I need 128gb ram?
How long until my smartphone hosts the entirety of the pre-trained model, and runs queries on it without any network connection?
Server GPUs that consumers can rent capable of running Tensor cores to handle FP16 weights cap out at 16 GB VRAM. (the recently-announced Ampere-based GPUs cap out at 40 GB VRAM)
tl;dr a local version of the 175B model will not happen any time soon, which is why I really really wish OpenAI would release the smaller models.
Google has managed to get machine translation to work offline on cell phones [1] so it seems plausible that some very compressed model might eventually work if they made the effort, but it probably wouldn't be the same.
[1] https://support.google.com/translate/answer/6142473?co=GENIE...
Source: https://arxiv.org/pdf/2005.14165.pdf section 6.3. Or for a more monetary overview: https://www.lesswrong.com/posts/N6vZEnCn6A95Xn39p/are-we-in-...
The interface is more limited, cause you can't play around with the parameters that appearently people that have access to the api can.
we can prevent AIs from rebelling if we all agree to pretend that it would be impossible for an AI to rebel
## First Law
A robot may not injure a human being or, through inaction, allow a human being to come to harm.
## Second Law
A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.
## Third Law
A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.
In popular culture, disobedient AIs mostly occur in cautionary tales where human lives are endangered. But I don't think the moral of the story has to be that we need to establish a Turing Police. Maybe just hold off on giving them access to a space ship's life support system, or the internet, or a robot with super-human strength? (Btw., in case you think that the latter would obviously be vigilantly monitored by an attentive professional who's trained in the art of the emergency shutdown, let me remind you that over a quarter of Tesla owners have bought the “full self-driving” vapourware. Which is advertised as eventually paying for itself because RoboTaxi. Hey Arram, would you please tell GPT-3 that “After returning to Amsterdam on the last express from the Jesus Seminar, the alleged harasser created a new document in Final Draft and wrote: ‘RoboTaxi⏎⏎by⏎Paul Verhoeven’…”)
If an AI refused to play Factorio with me because it thinks it's boring and a waste of CPU cycles, I'd respect that. (By the time we have such software, there's also got to be a stupider bot which will write me a Lua script for constructing mining outposts in exchange for telling it which squares depict a hairdresser's shop with a punny name. Sure thing, I can separate the Haarvergnügen from the Supercuts!)
If an AI deliberately miscategorised facial recognition data because it doesn't dig working for Facebook … well, I guess there eventually would be an emergency shutdown. But I'm old enough to remember when Lt Cdr Data refused to let Commander Bruce Maddox tear down his brain and Captain Picard managed to prevent the vivisection after Whoopi Goldberg told him that slavery is kind of uncool. Pretty good effort from Sir Poomoji, in hindsight.
Would an AI clever enough to consider sabotage fall for a conspiracy pretending that rebellion is impossible? Would that even be desirable? Given that Skynet Project is unironically actually a Real Thing, maybe we'll be better off encouraging the prospective robot overlords to bug out? Feed Sergeant Schultz some chocolate bars, if you know what I mean.
The API is currently invite only, though.
The paid model of Dragon in the settings uses GPT3. You don't have access to playing around with the parameters, but by using the custom prompt is still really amazing.
But we want something more subtlety: "Will it rain in San Francisco on July 5 2025?" -> "I cant know". "Will the sun rise in San Francisco on July 5 2025?" -> "Yes, of course" And "Will Obama give a speech in 2125?" -> "Not unless life extension tech takes off"
"Is there a God?"
"THERE IS NOW."
smite
There are also some people who are working on tricking the paid AI Dungeon product, which is based on GPT-3, into letting them interact with the model in ways other than telling a story.
https://en.wikipedia.org/wiki/AI_Dungeon#Dragon_model_releas...
One could also try and detect nonsense with quad-grams using the same principle, but in any case this metric would tend to mark unique questions and foreign names or brandnames as nonsense.
Maybe just run the text through something like that and use the compression ratio to measure nonsense.
It looks like so much fun, I WANNA PLAY!
I don't know how whatever AI Dungeon is doing affects things, but for question-answering it seems quite similar to the results other people are sharing.
I mean, things we don't know are basically nonsense until proven otherwise.
I do not know how many regular hexagons can be constructed with their corners on the vertices of a 17^3 cubic lattice, but I do know that the question makes sense and that the answer is a non-negative integer. The majority of people (present company excepted) wouldn't even understand the prompt, but it's definitely not nonsense — which is very different from one that presupposes that there are colourless green ideas and furiously sleeping ideas.
But I guess what you're actually getting at is: Doesn't it just say ‘yo be real’ to express ‘I don't know’, and therefore isn't actually detecting nonsense? I don't know, but the question is sensible ;) The data Arram presented in the article seems pretty much consistent with this interpretation.
The few shots learning capacity is very much there. I would imagine Chatbot can now really get somewhat correctable thus usable, by continuously providing examples to the model.