Hacker News Reader

Comment by starpilot | Hacker News Reader

I wish there was a way to augment data as easily for free text, and other business data. I always see these few-shot learning papers for images, I suspect because it's easy to augment image datasets and because image-recognition is interesting to laypeople. The vast majority of data we deal with in business is text/numerical which is much harder to use in these approaches.

View on HN

Agree with you on this. For text data, there was a paper called "UDA"(https://arxiv.org/abs/1904.12848) that did some work on this direction.

They augmented text by using backtranslation. Basic idea is you take text in English, translate that to some other language say French and then translate back the French text to English. Usually, you get back an English sentence that is different that the original English sentence but has the same meaning. Another approach they use to augment is to randomly replace stopword/low tf-idf(intuitively say very frequent words like a, an, the) with random words.

You will find implementation of UDA on GitHub and try that out.

I am learning these existing image semi-supervised technique right now and the plan is to do research on how we can transfer those ideas to text data. Let's see how it goes.

Haha that's how we used to generate blog spam content and comments :p

What are other techniques you use to generate spam? Maybe the research community can learn from you guys

Just replace a random subset of words with their nearest neighbors computed using a fancy word embedding model like BERT or GPT-2.

If you cannot do that due to them not including a vocabulary file, you're stuck using something like word2vec/FastText which is fine but not ideal if you're looking for grammatical correctness...

Yeah, since BERT has been trained specifically on this task of replacing random masked words, the resulting augmented sentence should sound natural compared to something like Word2Vec.

Reply on news.ycombinator.com