Comment by nmfisher | Hacker News Reader

Comment by nmfisher | Hacker News Reader

Parent Full thread

My problem is that Chomsky seems ardently committed to the "botanical" approach to natural language as the One True Way (TM).

By this, I mean the assumption that there is some universal logical structure to human language, and that our job is simply to come up with the labels for each constituent part and arrange them all correctly like a jigsaw puzzle. Once the puzzle is complete - boom! We've "solved" language.

That would have been a reasonable starting point back in the 60s, but I don't think it's advanced the field nearly as far as statistical methods.

Chomsky would retort that "advancing the field" is meaningless unless it allows us to "understand the field". But if natural language is a largely arbitrary collection of rules in constant flux, there would be very little of this "understanding" to be had. You could be bashing your head against the wall for centuries trying to induce structure into a bunch of symbols where none was ever to be had.

Now I do believe that purely statistical methods will hit a wall - the same way you could never teach a baby to communicate by throwing it a copy of Wikipedia and nothing else.

But if some parts of language are essentially arbitrary, then statistical methods are the best available tool to uncover that.

Language is infinite but the generating process (i.e. our brains or computers) are finite. The generators come in two flavors: rigid rule based (like Chomsky suggests) or stochastic (more modern view). The stochastic generators are obviously more powerful but difficult to "understand". Unfortunately, we associate the word "understand" with ability to have complete list of static rules. In my opinion, probabilistic rules are also as much "understable". Then there is dynamical probabilistic generators which I think our brains aren't evolved enough to truly "understand".

> Now I do believe that purely statistical methods will hit a wall - the same way you could never teach a baby to communicate by throwing it a copy of Wikipedia and nothing else.

Nick Chater and Paul Vitányi wrote papers[0] demonstrating that "the learner has sufficient data to learn successfully from positive evidence, if it favors the simplest encoding of the linguistic input":

- ‘Ideal learning’ of natural language: Positive results about learning from positive evidence,

- The probabilistic analysis of language acquisition: Theoretical, computational, and experimental analysis,

- Language Learning From Positive Evidence, Reconsidered: A Simplicity-Based Approach.

[0] https://homepages.cwi.nl/~paulv/learning.html

>> Chomsky would retort that "advancing the field" is meaningless unless it allows us to "understand the field". But if natural language is a largely arbitrary collection of rules in constant flux, there would be very little of this "understanding" to be had.

Imagine a historian of the future reading reams of code written for all the software we have in the modern era, in all the programming languages of the modern era, from Java to Brainfuck and back. Would the historian of the future be justified to conclude that there is nothing that connects these languages with one another, that there is no common structure, nothing that can unify them, "very little understanding to be hand"? If so the historian of the future would be wrong, because all modern programming languages are designed to run on the same substrate, modern digital computers. The existence of the digital computer is the unifying characteristic that explains the existence of any modern programming language.

In the same way, the variability of human language, internal or external, is without any shadow of a doubt explained by some processes in the human mind. These processes may be too complex to understand, perhaps, but understanding them is a worthy scientific goal. And there is, indeed, much "understanding" to be hand in that direction.

Reply on news.ycombinator.com