This, I suspect but do not know, may be a failing of the current direction of NL AI solutions. Selecting and using datasets to train NLP systems is a hard thing, especially if you truly treat it as a pattern to be learned rather than an enhancement of an underlying description of the particular language you're trying to learn.

Are we headed towards a time when deeply trained networks seem to work in an acceptably large number of cases, but how and why are unknown and susceptible to fatal flaws that rarely but catastrophically reveal themselves? I suspect that is the case with the current path of machine learning but hope I am wrong. It just seems to me that the inevitable results of certain efforts in machine learning are magic boxes that "work" but no one understands why or how, which smacks of the days of medicine prior to an understanding of the germ theory of disease where certain efforts at sanitation due to the belief in "ill humors" did improve health but were based on utterly false underlying theories, and those successes tended to reinforce other false solutions that did not improve health but in fact was harmful.