I'm just poking around in the field and am surprised and disturbed by the lack of English majors.
I was in the peanut gallery for a few planning sessions on reworking documentation at a major org and kept pitching the idea of hiring library science masters. Literally what they trained for and their other job prospects pay like $35k/yr. Shot down over fears that they wouldn't "understand the code", like that's the hard part of this job.
It's that English Majors don't understand how language works. Of course, neither do ML Researchers but their methods aren't contingent on understanding the problem space so that doesn't really matter.
Grammar is not how language works. It's useful fiction. English Majors have little to no "special insight" to give. When they do give "insight", it's more likely to derail the process than help it.
This is the conclusion after decades of trying it the way you imagine.
If you'd like to put the concept in a nutshell, talk to a high school teacher about your onboarding problems.
This has been tried for forever and never works. In fact, LLM's emerged from the *rejection* of traditional linguistics based techniques in favor of Moore's law based methods. In general, it seems that more general methods seem to win given enough data and compute (since the domain knowledge on complex problems are too difficult to encode into our programs and models). Getting English majors curious about mechanics and motivation of English is counter-productive because of their knowledge and flawed understanding of how language works.
Speak for yourself, but we're very much qualified to opine on this since we have a 70 year track record of having linguists in the room never working. In the same vein, I highly doubt high school teachers have any special insights into the most effective way to implement instructing LLMs or transfer learning.
> Speak for yourself, but we're very much qualified to opine on this since we have a 70 year track record of having linguists in the room never working.
70 years working next to another discipline and you didn't learn anything?
> In the same vein, I highly doubt high school teachers have any special insights into the most effective way to implement instructing LLMs or transfer learning.
The idea is to build analogies between the two domains so you can see how they approach similar or interesting problems. You're rummaging through Education's garage, looking for tools to steal.
You're the one who's supposed to have the useful insights.
Exactly. That's the whole point of what we've been trying to say. Linguists in the room aren't just unhelpful, but counterproductive. It's actively better that their wrong ideas of human language aren't included into models.
For a concrete example of this, see: https://en.wikipedia.org/wiki/Parse_tree
People have been trying to incorporate grammar tree's into models for forever and a half. Every time it's been tried it either is actively worse than letting the model come up with it's own representation, or slightly better at best if our model isn't complex enough.
>The idea is to build analogies between the two domains so you can see how they approach similar or interesting problems. You're rummaging through Education's garage, looking for tools to steal.
The whole problem is that there's more garbage in education to sift through than just coming up with new ideas by rummaging through DL literature. If you check the CS section of Arxiv, there's a million times the amount of ideas that haven't been fully explored and unlike linguistics or Education, actually have a decent chance of working. The whole point is that the field is 99% garbage for computational based applications, and researcher time is much better spent doing *literally anything else*.
A parallel and integrated system you wouldn’t be able to even deduce until things actually fail.