Hacker News Reader

What I've never quite gotten is, what's really the risk people are seeing? GPT-2 specifically I remember a great deal of handwringing (or hype) about how dangerous it was.

I feel like I even asked this same question here earlier: What's the danger? I hear about "polarization" and so on, but what's this supposed to enable that the bots and trolls and just good old regular people of today don't? Is it just a matter of scale?

A widespread ability to pretty convincingly fake more difficult things like photos and video seems much more relevant or "disruptive" than anything in the realm of text generation. I just don't really know what "fake" text does at the end of the day.

I rely a lot on text for obtaining information and shaping my opinion, and in many cases short form text plays an important role (e.g. here or on reddit). I’m sure I’m not alone in that.

This technology can at the very least waste my time, confuse me and hide the content that I’m actually looking for. It looks like it can feasibly generate 2-3 sentence comments that make sense in context, but in an automated way, with the purpose of injecting a specific sentiment into a comment section.

I already didn’t like that sometimes it seems comments I think are written by humans might not be (or they might not be sincere). This kind of technology can make that problem a lot larger.

It could flood the internet with so much crap, that is so hard to filter out, that the internet becomes a much less usable source for obtaining reliable information. I think that’s pretty scary.

Would you consider this comment to have less/no value if you found out that it was generated by a bot? What if quality and information density of automated text surpasses human contributions? Will it still be just spam?

It depends. At the moment if I see a reddit post saying product X was really appreciated by a user, most of the time I'll believe that was an actual human appreciating that product. But if modern mass marketing is going to be the injection of seemingly sincere product recommendations into reddit threads that will obviously lose value - the bot comment is lower value than the human comment, and because I can't distinguish them all such comments lose value. Similarly for political statements of support.

I'm sure there is potential for extremely useful bots (e.g. such as article summarization bots on reddit) which increase information. I guess it really depends on who decides to set up a bot, and their goals and implementation.

Many people have no clue that automation has come this far and will judge every comment they read online as sincere. If they're actually not, and many are driven by political and commercial agendas, I think that's a bit dangerous, because people will act on them.

Software engineers naturally focus on what's a new capability for software. GPT-3 looks nearly capable of shoveling out the sort of low-quality content that's primarily curated by other algorithms, like clickbait news or content marketing, which was still firmly beyond software capabilities last year.

Software engineers also frequently underestimate what can already be done with a heap of poorly paid workers. So while it's a new capability for software, it's not a new capability for business. You're right that this can already be done with people.

Have you ever hired human text writers?

I have. And I can tell you even with not-so-poorly-paid people you often get craptacular results. Writing is really damn hard, especially writing something which require domain knowledge.

From examples I've seen, it looks like GPT-3 works a lot better than your average human writer, for a fraction of cost, for a fraction of time.

A lot of low-quality content you see on the web is more of a copy-paste job than actual writing. Arguable, GPT-3 is also doing a transformative copy-paste, but it remixes content of thousands of sources in a coherent way. Most people can't do that.

Indeed, any modality that AI can perfectly imitate decreases trust in said modality. Visual modalities are more critical as they are more difficult to fake.

Though to do significant damage in this manner, you'd probably need to hold all media as hostages via mass scale social engineering. But at that point, trustworthiness of the news is a minor concern among all AI safety issues. Then the AI can literally wipe clean the entire planet barring a small elite using via nanobots, bio-engineering and drones. This is a highly likely event as there is a very strong incentive to prevent another AI from doing the same first.

Here's a paper I co-authored about the broader implications: https://thoughtfultech.org/malicious-use-synthetic-media-mac...

There's a bunch more out now.

This might also be helpful: https://medium.com/@aviv/what-does-a-world-with-automated-so...

We must return, regretably, to judging the quality of a text on the content within itself, instead of by appeal to the authority who wrote it. This will all change if we can agree GPT-{3,4,...,n} is the greatest author on this planet.

To be honest I find it hard to believe that won't be the case within this century.

Right, when judging an individual piece of text you should evaluate the merits of the argument. Where this becomes dangerous though is in building what looks like consensus among thousands of users of a service.

Text like this makes it feasible to build huge puppet networks that can get upvotes/karma from real users quite easy by playing into their echo chambers.

This type of technology could easily be the undoing of anonymous upvote driven forums like reddit/hn.

> Now we can't trust text, the most trusted medium in human history, and then what?

I don't understand what you mean by this. Isn't text the least trusted medium? Anyone can easily try to impersonate anyone else in text. Going by your argument of trust, shouldn't we be more afraid of deepfake videos? Or even more accurately, deepfake videos paired with something like GPT-3.

Yes, we should also be afraid of deepfake.

With GPT-3, the issue isn't so much being able to impersonate somebody, but the ability to generate human-seeming text at scale. This allows you to create a false perception of public sentiment by maintaining fake accounts on online forums, writing fake letters to the editor, to politicians, and so on.

On the flip side, this is already possible today with content farms, and perhaps GPT-3 can save us by being the thing that finally erodes our trust in these signals.

One thing is sure though, it drives yet another arms race, and arms races are always a net loss for society.

The cat is out of bag already -- people know what one can accomplish if you scale up a GPT-2-like model. OpenAI has no control over it -- anyone with enough hardware and a bit of knowledge can make his own GPT-3, perhaps specialized for his needs.

The scariest thing is that GPT-3 is completely unoptimized. If you want to generate indistinguishable articles in a particular field, you can o it with fine-tuning, or, perhaps, adding GAN on top of it.

If society put a lid on OpenAI do you think others are going to stop doing language model research? Luddite responses don't fare well with new tech. Society will adapt, there is already a lot of garbage text online, and we'll find uses for LMs.

Nah, I think this might look worse than it is to "always online" types. And sure, it might be the death knell for Twitter and reddit. But it fortunately has zero impact on face to face interactions. I guess the 90s, before social media, before going to the internet for entertainment, before Yelp and reddit for buying advice, were before many commenters here were aware, but they weren't so bad. A reversion to that time period, but with today's ability to video chat with friends and family and the compute power available for, say, drug discovery, doesn't sound so bad to me.

In fact, I'm struggling to think how GPT-3 affects me at all, given my lifestyle.