You can make any reasonably flexible tool do awful things. Anybody can go open up Word and write a horrific, racist screed in it. That doesn't mean that Word is racist, it means the person using it is racist. If Word took my book report on Narrative of the Life of Fredrick Douglass and used autocorrect to add a bunch of racist stuff, that would be an actual problem with Word.
The same is true with LLMs - if you ask it an anodyne question, and it comes back with something racist, that's a big problem! If prompt engineer it to think that you're a history professor trying to get examples of racist things that white people might have said to black people in the past for an important paper on historical racism, and then it says something racist, that's not an actual problem with the LLM. (If you then go take it and post it on a message board or what have you, then it is a problem with you.)
But the more complicated aspect is that of mode collapse. A good example of this is with Alpha Go's game against Lee Sedol. The game that Sedol won was weird. Alpha Go played in a very weird way, and a very bad way. This is due to the probabilistic nature and that it had likely wandered into a latent space that it had not seen before. Remember that it was mostly trained against good games and so can easily get confused in bad games (as Sedol also got confused due to its previous high performance and he thought it was playing some "5-D chess"). This compounds with the above aspect as now there exists a mechanism wherein such toxicity can arise through seemingly unprovoked means. We call these hallucinations btw.
Of course, this does mean there is good reason to study said toxicity. But we shouldn't conflate academic curiosity with theater. Are people uninformed and taking AI risk too far? Most certainty. But the same can be said about the hype of these tools too. Both are clearly detrimental to the progress of AI advancement. In this we probably should be a bit more measured in our quickness to respond and critiques. Tribalism doesn't belong here but people will encourage it.
If you can take enough of them to task on social media, you can influence public discourse. The slow boil cooks the frog.
Having no place for free constructive open discussion, society is bound to not only stagnate but retardate. Parcellation into echo chambers only exacerbates the problem.
I'd bet my hat that a non-trivial number of posts here on HN are, and have been, generated via GPT-3, and before ChatGPT became big. There are definite signs of astroturfing, and you can almost predict which threads are going to have it -- China, Tesla, some BTC discussions.
They don't even need to be convincing, just present and in enough volume to drag down or derail discussions; "The Firehose of Falsehood" model.
While the underlying model is certainly different, and my understanding is that current LLMs don’t learn “live”, the principle seems worth keeping in mind.
They aren't really teaching the model anything it hasnt seen before.
If you want toxic output, you can get it.
Just seems like pearl clutching at it's finest.
Filtering shitty content is easier than creating it with a properly constructed LLM system, the complaints about toxic outputs seem to me to be analogous to an electrical engineer complaining that the voltage from the mains is wrong for their device, but refusing to google what an (electrical) transformer is.
Toxic writing pre-exists LLMs. LLMs output writing. This is not a new problem, but we have a new solution - LLM filtering.
Can you explain this? It feels completely wrong - even OpenAI, who probably have invested the most, can't filter out all "shitty content". An LLM can on the other hand create "shitty content" incredibly easy - even if the creators try to stop it! So how is filtering easier than creating?
People keep pretending that the LLM is some kind of "natural" giving, like a periodic table or something. No, LLM is created by humans. A species known for their limitations and biases.