Secondly, have you seen the results? I was dumbfounded and fascinated. I spent hours reading the samples.
Maybe I'm just out of the loop and this truly isn't anything significant, but then that only proves that OpenAI was successful: Now I am aware of the latest advances in NLP and hopefully so too are many more.
I wouldn't be so sure about that. Take their reporting of Charlottesville events and Trump's comments about them. Here's what Trump _actually_ said: https://twitter.com/ZiaErica/status/1096572062196486144. Pretty reasonable point of view, all things considered. What was NYTimes "reporting"? That Trump is "defending white supremacists", of course. Don't believe me? See for yourself: https://www.google.com/search?q=trump+charlottesville+nytime.... Why was NYTimes doing that? It's either deliberate malice or incompetence, both of which would make NYTimes quite friendly to automatically generated fake news as long as they fit their narrative.
But there's a bigger issue with all of this. When people see this tech, they immediately think that it'll be used to generate fake news (which it will be, to be sure). BUT, it could also be used to do the exact opposite: take facts and summarize them without agenda-driven omissions, without "reading minds" or inventing "sources" "familiar with" someone's "thinking", or passing off uncorroborated dossiers or book chapters as gospel truth.
Yes, I've seen the result. They're nice but, as the article points out, not extraordinary compared to state of the art, open NLP research.
OpenAI's behaviour here smells of Gibsonesque 'anti-marketing', using the misunderstanding of AI and its capabilities in the general population as a means to stir up publicity for their organisation.
This is unethical, misrepresents progress in the field, and produces confusion in the press.
> misrepresents progress in the field
Can you point me to some examples of unsupervised learning with similar results? Not asking for rhetorical purposes; I just genuinely was shocked by how compelling their results were, especially given this was unsupervised.
> OpenAI's behaviour here smells of Gibsonesque 'anti-marketing'
I don't disagree that the ethics are questionable, but I think it's highly speculative to suggest that they didn't release the full model purely as a marketing ploy (I'm assuming this is the main objection to their marketing "tactics"). As you say, it "smells" this way, but I fail to see how it's really so clear-cut.
Model wise this is just openAI's GPT with some very slight modification (laid out in the paper).
Ilya has now commented in the thread and essentially made the same point, this is state of the art performance, but reproducible by everyone because it uses a known architecture.
The secrecy and controversy makes no sense if the model is open, even the methodology of data collection is laid out. There is no safety here assuming that anybody who wants to rebuild the model can do so simply by putting enough effort into rebuilding the dataset, which is not an issue for a seriously malicious actor.
> The secrecy and controversy makes no sense if the model is open, even the methodology of data collection is laid out.
This is exactly why I found the results so compelling: It suggests that this technology is already accessible to some big players: The odds that a Big Corp. or govt agency has already begun using the technology are high, which is precisely why the public needs to start thinking about it.
I cannot know exactly why OpenAI chose to withhold the model, especially given how easy it would be to recreate, but even if we assume that OpenAI withheld the full model purely to drum up controversy, the controversy is justified, as it's very likely that this technology is already in the hands of a few big players.
If they did, I bet it would be used for automated "troll farms".
Like weaponized malicious ELIZA, it would have fake user profiles reacting to keywords, spinning suitable counter-argumentation and/or lies for as long as it takes to change opinions and perceptions, relentlessly, day and night.
This isn't my impression.
It's not the best in many domains but it a single network that is moderately decent in many domains. You can use it to summarize by adding TLDR to the end of text. You can use it to translate by listing previous translations. And of course it does blow away any state-of-the-art RNN text generation I've seen. RNNs tend to fall apart after 1 or 2 sentences where as this holds it together for multiple paragraphs.
I'm trying to get a sense of just how quickly things have been advancing. I read a few NLP white papers about a year ago and never saw anything as compelling as this, but I am definitely an outsider possibly on the left hand side of the Dunning Kruger graph...