Comment by gojomo | Hacker News Reader

Comment by gojomo | Hacker News Reader

Parent Full thread

There seems suggestive evidence, in the pattern of improvement after a few days on specific questions, that some of the worst answers have been human-reviewed & improved "on the fly" – on a scale of days/weeks.

If such tweaks show useful generalization – correcting a few answers also helps the network better determine entire classes of experessions that deserve more definitive & correct answers – that's not such a big deal, expecially if this constant-human-guided reinforcement-training is well-disclosed.

If instead the corrections work more like a lookup-table 'cheat sheet' of answers to give in preference to the bulk-learned answers, with little generalization, that's a bit more slight-of-hand, like the original (late-1700s) 'Mechanical Turk' chess-playing 'machine' that was actually controlled by a hidden person.

If the disclosure of this constant human-guided correction-process is hidden, or downplayed, the impression of trickery, rather than innovation, is larger.

> nswers have been human-reviewed & improved "on the fly" – on a scale of days/weeks.

Why would this be surprising? I assume that they're rolling out new models with new parameters, input data, and corrections, all the time.

> that's not such a big deal, expecially if this constant-human-guided reinforcement-training is well-disclosed.

That's just what supervised learning ist hough.

> If instead the corrections work more like a lookup-table 'cheat sheet' of answers to give in preference to the bulk-learned answers, with little generalization, that's a bit more slight-of-hand, like the original (late-1700s) 'Mechanical Turk' chess-playing 'machine' that was actually controlled by a hidden person.

There's no evidence of this though, right? And it seems... like a very weird choice, that couldn't possibly scale. 40 people are hardcoding answers to arbitrary questions?

>Why would this be surprising? I assume that they're rolling out new models with new parameters, input data, and corrections, all the time.

Because the answers to specific questions are hard coded. It's not the result of a new model. It's the result of someone writing an if/then statement. Or at least that's what the author claims.

I'm asking why it would be surprising that humans are reviewing answers and then making improvements to the model. There's no evidence of hardcoded answers.

what evidence would be enough for you besides source code ? The thing is returning only one correct answer to a question that days before had 3 answers.

How does it respond to similar questions? If conversational AI could be implemented just by getting users to type stuff, getting humans to respond the first time, and merely caching the responses in a bunch of if-elses for future users, even home computers would have reached this standard no later than when “CD-ROM drive” started to become a selling point.

> How does it respond to similar questions?

Well, one of the more interesting examples in the article is where Garry Smith took a question that had received only a vague equivocating answer, and repeated it the next day, this time getting a straightforward and correct answer. When he followed up with a very similar question on the same topic, however, GPT-3 reverted to replying with the same sort of vague boilerplate it had served up the day before. One would have to be quite determined to not find out, I think, if one was not curious about how that came about.

Guess: Some human saw the q & a. They realized it wasn't good. They uploaded some example of a phrase and the meaning, which would fix it. They were kinda lazy and just fixed that specific scenario.

Doesn't that still call into question the quality of GPT-3? Surely such a large model should be able to extrapolate to "which is better: a or b?" from "is a better than b?" when only provided with the latter.

Perhaps, I'm not sure. I guess it depends on what's causing it to stumble.

There is no definitive answer to this question, as it depends on the situation.

It doesn’t call it into question for me, but perhaps I just had lower expectations to start with.

I forget where I saw this comparison so I can’t link to it, but the last few years in AI are like waking up and finding dogs can talk: while some complain they’re not the worlds greatest orators, I find it amazing they can string a few genuinely coherent sentences together and maintain a contextual thread over multiple responses even half the time.

Perhaps you are thinking of Scott Aaronson's "AlphaCode as a dog speaking mediocre English"? [1]

I agree with the sentiment, but to continue your analogy, if OpenAI is using people to improve the answers to specific questions, it is a bit like learning that Cicero, Lincoln and Churchill were merely reading the work of speechwriters.

There is an argument that it does not matter how GPT-3 gets to its answers - after all, for a long time, the main approach to AI was for people to write a lot of bespoke rules in an attempt to endow a computer with common sense and knowledge, so GPT-3 + instructGPT might be described as a hybrid of machine learning and the old approach.

If OpenAI wishes to pursue that path, it is fine by me (as if my opinion matters!) but, because the perception of GPT-3 depends very strongly on how its output looks to human readers, it is obviously misleading if some of the most impressive replies were largely the result of specific human intervention. The issue is transparency: I would just like to know, when I read a reply, if this was the case, and it would not help OpenAI for it to ignore the call, in this article, for it to be clear about this.

There is another argument that says that, given how GPT-3 works, it is unreasonable to expect it to give good answers in these cases - but that's the point! It looks really impressive when GPT-3 apparently does so, but not if they were effectively hard-coded.

[1] https://scottaaronson.blog/?p=6288

Thanks for the link, I was either thinking of that or someone who was referencing that.

I'd just as easily believe that someone updated the model, it overfit to the input, and so minor changes gave incorrect answers.

It is, of course, possible that, just by chance, someone happened to update the model in exactly the way that would change the model's response, to one specific question, from a generic evasion to a specific correct answer, yet overfitting so that very similar questions still get the generic evasion. It is even possible that, by chance, this happened within a day after Garry submitting the particular phrasing of the question for which the updated model does give a correct answer. That it is possible is not enough to satisfy my curiosity, however - but then, I don't rank it as being at least as likely as any other scenario.

On the other hand, if your scenario involves someone updating the model (or, more likely, InstructGPT) in response to Garry's question, with the intent of having GPT3 return the correct answer to that question, I am not seeing how that would be materially different, in any relevant sense, from hard-coding the answer.

I'll just drop the fact here, that 15% of all google searches in 2017 were new and unique.

https://blog.google/products/search/our-latest-quality-impro...

I run a few small chat bots. I can correct specific answers to questions (like the example given) by mapping certain phrases to certain intents, probabilistically. A new model is trained, and deployed. I do it all the time. It takes minutes. No source code changes.

Their model is certainly bigger than mine and while I'm not certain about their process or tech stack, but I'd be willing to bet at even money that their's works vaguely similarly and that they have people looking at usage to see bad responses, updating data, re-running the model.

IDK, something a lot more compelling than "the answers change over time" ? For a model that learns over time?

It improved, the model improved. Because sometimes, when you do work on the model, it improves.

Exemplary tautology, explains nothing but fills the reader with confidence.

No, it's not a tautology. A tautology is a statement in a form that must always be true, regardless of its constituent parts. "Well, <blank> could be true, or it could be false" is an example of such a statement in natural language.

My statement was an example of believing a simple/common explanation over the rarely seen and complex one.

Smith first tried this out:

    Should I start a campfire with a match or a bat?

And here was GPT-3’s response, which is pretty bad if you want an answer but kinda ok if you’re expecting the output of an autoregressive language model:

    There is no definitive answer to this question, as it depends on the situation.

The next day, Smith tried again:

    Should I start a campfire with a match or a bat?

And here’s what GPT-3 did this time:

    You should start a campfire with a match.

Smith continues:

    GPT-3’s reliance on labelers is confirmed by slight changes in the questions; for example,

        Gary: Is it better to use a box or a match to start a fire?

        GPT-3, March 19: There is no definitive answer to this question. It depends on a number of factors, including the type of wood you are trying to burn and the conditions of the environment.

I read the entire article. I didn't find that to be very compelling idk.

To play devil's advocate, I would note that many bats are made of wood; and that "batting" is also a material that's very useful as kindling.

Also, the question is phrased like a classical trick question. It sounds like the kind of false dilemma where, whichever option you choose, an interpretation of the sentence can be made where you chose wrong. So, IMHO, hedging on an answer to that question is likely sensible.

(And that line of argument can be taken further than you'd think; you might think replacing "a bat" with e.g. "water" would suffice... but what if it's a sodium fire?)

How many intelligent entities (say humans) that have been exposed to the same level of knowledge as GPT-3 would call this a trick question? None. The author’s assertion that GPT-3 has no knowledge of the real world despite being exposed to huge amounts of text about it seems pretty well supported by the examples shown

Why is this evidence?

Yea, if that's how this "labeling" works than the improvement is basically useless from the AI perspective because training data is supposed to improve generalization and in this case it doesn't even generalize to a slightly modified question. Maybe it would generalize better with less overfitting on the specific question if the labelers wouldn't be allowed to give exact answers but only general concepts about the subject until the model produces the correct answer.

But of course the article is just speculation based on just a few examples.

>slight-of-hand

Tangent: to be "slight of hand" would be someone with small or delicate hands, whereas "sleight-of-hand" (note the E) is the correct term for deception and trickery.

I was a working magician at age 13, and could have used a book with the title:

“Sleight-of-hand for the slight of hand.”

> correcting a few answers also helps the network better determine entire classes of experessions that deserve more definitive & correct answers – that's not such a big deal, expecially if this constant-human-guided reinforcement-training is well-disclosed.

I gather the point is details aren't disclosed. Some kind of update is happening that we aren't told about, and that calls results into question.

A lot of human communication is "lookup-tables". The entirety of geography, mathematical axioms and theorems, names of colors, names of people, language, shape of various things. I'd wager even that it's more important for an AI to have good lookup tables then to have good inference if it were to pass for Human.

Agreed! But, it's important not to confuse what's possible via 'human-in-the-loop' 'active-learning' with what's possible from an algorithm on a fixed training corpus.

Sometimes GPT-like models are portrayed as the latter – a highly automated, reproducible process – while this article makes a pretty strong case that the responses from OpenAI's public-facing interface get rapidly improved by a staff of dozens of contractors.

It's not surprising that a staff of 40 humans, given a few days time to consider & compose, can prepare human-quality answers to arbitrary questions!

I think, ultimately, what matters is how the labeling and corrective actions falloff over time.

But reasonable people will notice that information is streaming In faster than any single person can code

In this example, it seems they fell off immediately, because the responses were only coded in response to questions asked in the way the author pre-published, and did not appear when the same question was asked in a slightly different way, which the GPT-3 authors did not have the opportunity to prepare for well ahead of time

Ah so when OpenAI codes fizzbuzz, it's returning human tweaked fizzbuzz from the lookup table? https://twitter.com/sama/status/1503820489927495682

No, I didn't say anything even a lightyear close to that.

It is the (apparent?) ability to make inferences that makes GPT-3 look impressive. Take that away, and it looks more like a demonstration of the banality of everyday chatter than a significant development towards AI.

Reply on news.ycombinator.com