228 karma · joined August 9, 2022
To summarise it quickly, Chomsky's contention was that all the world's languages can be described by shockingly few degrees of freedom on the same universal grammar, and that we learn language surprisingly fast relative to training data because all we are really picking up are those parameters and the rest is hard wired from birth the same way horses come out the womb already hard wired to gallop.
Decades later, very few things have truely stood the test of being universal among languages, but it was still a valuable contribution because he poked a serious hole in the pure Hebbian reinforcement theories which were in vogue back then.
There is something fundamentally flawed in the approach not in the data.
There are things the brain does we have not yet been able to reproduce with a neural network, or to the extent we have seemingly with excessive resources of training and network size. Therefore there is some salient feature of neurology which has been overlooked. I don't think it is necessary to mimic biology down to the exact function of real neurons, but there must in fact be something we are neglecting to mimic.
I know I did. Until one night I randomly pursued the place in street view. It's uncanny, looking at it, not being sure which things changed and which you don't remember correctly. Was the swingset in the park always yellow? Was that store always on that side of the other one? It really fucks with your sense of reality