Comment by jasode | Hacker News Reader

Comment by jasode | Hacker News Reader

With GPT-3, I guess lots of upcoming stories of various discussion forums getting the "Sokal Affair"[1] treatment. We'll keep amusing each other by trolling everybody with more fake GPT-3 stories.

I think GPT-3 is very convincing for "soft" topics like the other HN thread "Feeling Unproductive?"[2], and philosophical questions like "What is intelligence?" where debaters can just toss word salad at each other.

It's less convincing for "hard" concrete science topics. E.g. Rust/Go articles of programming to improve performance.

An interesting question is what happens when the input to the future GPT-4 is inadvertently fed by lots of generated GPT-3 output. And in turn, GPT-5 is fed by GPT-4 (which already ingested GPT-3). A lot of the corpus feeding GPT-3 was web scraping and now that source is tainted for future GPT-x models.

[1] https://en.wikipedia.org/wiki/Sokal_affair

[2] https://news.ycombinator.com/item?id=24062702

> A lot of the corpus feeding GPT-3 was web scraping and now that source is tainted for future GPT-x models

It might be possible to filter out GPT-3 generated text from future training data. Simply feed part of the text to GPT-3, and if it is way too good at predicting what follows you throw it out. The same trick could be used to detect students writing essays with it.

This trick will stop working though as more variants of good text prediction algorithms appear, unless we can do the same test against each one.

If you wanted to generate a longer essay by GPT3 but make it really hard to tell, you could just get it started with a 1-2 sentences. Then let it generate N-1 tokens (where N is the window size, I guess 2048), and write a few tokens yourself (and repeat). Then I don't think there's any way to reverse engineer the internal state of the model at any point in the text. It would be like trying to find a string that hashes to a given value.

Maybe like hashing though there can be weaknesses in the algorithm still.

There is one fringe ironic and probably unrealistic failure case with that approach - what if GPT3 matches expectations too well with actual people and results in good examples being tossed in preference for posts which don't fit with most like say blatant spambots?

But only the best examples of GPT-3 output will be worth putting on the web, so even without any code improvements in future GPT-x models you can still expect an improvement in quality. A writer (human or AI) should be able to improve by writing a million articles and then be told which ones are good (i.e. worth publishing).

Not even actual treatments of soft topics. Take this essay. It’s a solid C paper.

So, mostly just Internet forum philosophy.

It was asked for internet forum philosophy, not academic philosophy.

Reply on news.ycombinator.com