Which leads to a simple improvement: teach the model when to stop, what sections to remove. There’s already great ML-based summary engines, and systems able to answer questions from descriptions.
Which leads to a simple improvement: teach the model when to stop, what sections to remove. There’s already great ML-based summary engines, and systems able to answer questions from descriptions.
GPT-3 reads like the unfiltered garbage that comes out of my head when I'm trying to get ideas out. The primary difference is that I have empathy for the audience so I go through the effort to reread, clean it up, clear it up then cut it down.
But that secondary editing process has nothing to do with the subject matter or initial content generation process. It was trained through years of *not* doing it and having my audience ask lots of questions and clearly misunderstand what I was trying to say (let's call it reinforcement learning). As a result I now have an intuition about which concepts need explanatory analogies, based on my understanding of what the readers know, and which are almost certainly going to be understood outright. I think this gets at the root of the difference.
Basically when I write, I'm not trying to generate text, I'm trying to generate training data that will amend foreign models with a high fidelity representation of a subset of mine.
This feels GAN-like but different. It's like you need to have two separate models, the 'expert' and the 'n00b' (or ten n00bs trained on different data), ask both of them the same thing, and one-shot retrain the n00b from the answers provided by the 'expert' until the n00b is able to generate content that the expert finds to be sufficiently similar.
Well said! Replying so I can find this thread again, please ignore.
People who are afraid of the current generation of ML writing are absolutely dogshit writers with terrible taste in writing. I keep forgetting who said this, but theres a saying to never start writing by describing the weather. Lots of folks miss the point to this. You could modernize it by saying, "Dont start talking about your childhood when writing a recipe." Why? Because no one fucking cares. They want the recipe. Not a description of your childhood or what the weather was like. Take the time to notice articles and "journalist writing". If it starts with the weather, notice how incredibly long it takes for the author to get to the point or describe anything functionally meaningful. You'll notice these articles are extremely long in word count and extremely low in actual information/content. These are the people afraid because yes, a machine can keyword drop and fluff better than them.
Is it possible that there will be a future AI/ML to write with purpose? Maybe. But realistic idea connection and logical flow is still a huge hurdle for AI/ML. Summary engines seem good on the surface, but again, if you actually read them for a real world reason, not for "look how good ML is", you'll notice they're as meaningful as a politician running for office. Lots of keywords, lots of nothing really said or done... just like anyone with a liberal arts degree.