I think we may agree more than you realize, but perhaps we aren't clear about what we mean. As I said pretty early on - they are similar at some level. To make an analogy, that could be as simple as a submarine (w2v) and a fish (authentic language) now 'uses fins' to move (bert). They're now more similar, but not the same. Of course there are differences, but we can learn by studying what structure has been added and how in order to work towards better theory (as well as looking at what differences there are). I have read all of these studies, as I am an academic in related fields, and you are correct that the resolution is not as good as we would hope (it's better in the ventral visual stream in non human primates, which is one reason we started there); however it doesn't negate the importance of the predictive power here. We can make very useful tools out of these models which can improve the lives of non-verbal people for example.

Another interesting set of works going in recently is in testing out the poverty of stimulus. There is now evidence from many labs that LMs learn with the same amount of data as humans, so the sample efficiency arguments and the 'poverty of stimulus' aren't really as successful as arguments for training difference either.

To the point of humans not learning certain languages, I haven't seen it proven that humans are absolutely incapable of learning certain languages. I do know that children ignore much of sequence structure and create their own internal structures, but this does not negate learning languages which have some arbitrary set of rules. I'm aware that models learn DNA more easily _practically_, but this is more of a statement about human experience and various other environmental factors - not a mathematical proof that it's impossible for humans to learn. Furthermore, that is a nice property of these systems. If they are capable of modeling many different phenomenon, they are useful for many things. If they can settle on a model configuration that linearly maps to a similar activation space with similar topological properties to a patient, that's also very useful. In other words, If the topological properties of the activation space are more constrained in the cortex vs more flexible NNs, but the NNs can fit to the constrained space, that doesn't seem like an insurmountable problem in studying either trained system.

Ultimately, I think we can agree that there are differences, but there are also striking similarities that can be very useful for improving science and medicine going forward, and perhaps (with _substantial_ effort in linguistics and mechanistic interpretability fields) we may be able to improve some of our understanding of linguistics and/or neuroscience (or perhaps not, but it's at least a promising potential lead).