You make good points, but this one isn’t quite what I meant. I didn’t mean we are trained by evolution for a particular language, but that evolution selected for a language skill. In other words, we are “pre-baked” with the ability to learn a language. It would be analogous to a DL model being trained for generalized regression but not a particular problem. To that extent, I think we’re saying the same thing. Some of the theories related to our generalized learning abilities postulate they stem from this base ability (our aptitude for music, for example, being a consequence of our language learning ability)
>I see BERT as non sensical. You need to be scientific and have a mathematical theory
This is a matter of contention. A lot of science progresses by starting with an empirical result that drives a change in theory rather than the other way around. I can’t remember who to attribute it to, but there’s a quote to the effect that “‘Thats odd’ is the most productive phrase in science, rather than ‘Eureka!’”