Well, no, it can produce a probability distribution over all possible tokens, among which ‘yes’ or ‘oui’ or ‘はい’ or ‘totally’ or ‘no’ or ‘nein’ or the beginning of ‘as a large language model I am unable to answer that question’ are all represented. Which is either more or less impressive than just being able to answer ‘yes or no’ depending on your priors I guess.

There’s maybe an interesting philosophical question of perspective there because if you think of the GPT as answering the question ‘if you had just read this, what token would you expect to read next?’ That doesn’t seem like a question that necessarily requires ‘intelligence’ so much as ‘data’. It’s just a classification problem and we’ve been throwing NNs at that for years.

But if you ask the question ‘if you had just written this, what token would you expect to output next?’ It feels like the answer would require intelligence.

But maybe they’re basically identical questions?