Creating Artificial Mechanical Turks with Pretrained Language Models

unite.ai

25 pointsHard_Space6 comments

How far are we away from using open.ai to earn money on Mturk? Or is this like the fusion problem?

Would this even be fraud if it worked, or is it the expected outcome of this sort of market?

tldr: using large expensive models to auto-label data to train small cheap models.

(I find the 'mechanical turk' framing here to be much more confusing & misleading than clever or helpful, and to make it harder to compare to the considerable number of other papers on using language models to generate new datasets & do self-distillation.)

>to make it harder to compare to the considerable number of other papers

Naturally. There's a reason AI papers are not published in respected journals a significant proportion of the time.

Isn't this just "transfer learning"? Surely there has to be a better way than "momma bird pukes into baby bird's mouth" type of training

No. Transfer usually means using the same NN model (eg. GPT-3 checkpoints being retrained on Github and then called 'Codex'), or possibly some sort of distillation/sparsifying approach. This is about auto-generating training data, maybe not even meant to be used by a neural net at all.

Reply on news.ycombinator.com