Hacker News Reader

Comment by fauigerzigerk | Hacker News Reader

Let's consider the supreme court handshake problem, but say there is an unwritten social law that forces judges to only ever initiate a handshake with judges less senior than themselves. If the seniority of two particular judges happened to be exactly equal, they would not shake hands at all.

Let's assume (I don't know if it is actually true or not) that in the history of the supreme court, there have never been two judges of the exact same seniority. In that case, a model learned from handshake data would not include the slightest hint of this unwritten social law.

I think what Chomsky is saying is that if we do not understand the generative principle behind any data, we cannot possibly know what circumstance might completely invalidate our model. There may not be a way to smooth this out.

Language understanding, contrary to things like speech recognition, does not lend itself very well to smoothing.

View on HN

The way Norvig tells it, language lends itself perfectly well to smoothing and even Chomsky's favourite absurd phrase "colorless green ideas sleep furiously" is assigned a (very small but non-zero) probability in some statistical models of English.

Truth is you can't do much with statistical models of language without some sort of way to account for what's missing from your data, which is always most of language. On the other hand, anything you might do is never going to be enough when that's the case: that you're missing the majority of language from your training data.

Some NLP tasks do indeed lend themselves well to smoothing, but I was thinking of language understanding tasks like question answering where changing or misunderstanding a single word in an entire novel can easily make a correct answer completely incorrect.

I agree with your second paragraph, and if I understand Chomsky correctly, that is part of why he argues in favor of a generative grammar. I can't say that I completely understand how such a grammar would be linked to semantics and experience though.

Ah, OK, sorry for misunderstanding.

Was that a pun?

Out of curiosity, how does Chomsky's generative model account for language understanding?

It demonstrates that human languages all have a certain structure, so it must be an innate faculty of human beings (as opposed to acquired). A neural network could not tell you that. It also suggests further avenues of inquiry (Why that particular structure?)

I think you misunderstood my question. Here's my understanding of Chomskyian linguistics:

1. You have a certain concept you wish to express.

2. You apply a generative grammar to the concept, producing a linguistic statement.

3. You express the statement in a linguistic performance.

4. I perceive the linguistic performance.

5. ¿I reverse the generative grammar to produce the concept?

6. I understand the concept.

My understanding is that Chomsky is only interested in steps 2 and 5 (and that he is explicitly uninterested in 3 and 4). But how does step 5 work?

I don't think much is understood about how linear externalizations of language are deserialized into symbolic structures, what those structures are, and how those are symbol structures are mapped into mental representations.

Hah. No pun :)

The issue Chomsky raises isn't that statistical models aren't predictive enough, but that you don't understand they don't typically yield any understanding of the underlying system.

Similarly (and I understand this argument went through physics long before quantum mechanics showed up and made the argument seem silly), the ideal gas law, say, is predictive, but doesn't yield any understanding of the underlying system.

> If the seniority of two particular judges happened to be exactly equal, they would not shake hands at all.

Somewhat similar to Buridan's Ass.

... an ass that is equally hungry and thirsty is placed precisely midway between a stack of hay and a pail of water. Since the paradox assumes the ass will always go to whichever is closer, it will die of both hunger and thirst since it cannot make any rational decision to choose one over the other.

https://en.wikipedia.org/wiki/Buridan%27s_ass

If this is unwritten (I take this to mean a piece of culture people practice without ever formalizing), how would the judges deal with the novel scenario? That rule will not have existed until it starts being practiced.

The judges would simply apply the existing rule as it exists in their mind, but there is no trace of that state of mind in the handshake dataset.

Yes the machine could guess. It could even guess correctly. But I think what Chomsky means is that we have no valid scientific reason to believe that it would.

If the machine is faced with this novel scenario before the judges, guesses, then that becomes the cultural norm.... does the machine win?

> I think what Chomsky is saying is that if we do not understand the generative principle behind any data, we cannot possibly know what circumstance might completely invalidate our model. There may not be a way to smooth this out.

No, what Chomsky is saying is that if you don't know the generative principle behind the data, then you don't know the generative principle behind the data. You don't understand a system just because you can faithfully predict it.

Can't you learn this rule by generating (simulating) the behaviour with various hypotheses?

So you'd have a model where handshakes are dependent on the relative heights, and it would fail because some senior judge who is short goes around and initiates.

And then you'd come to the correct one, if the data is there for it. Like a hypothesis that's never refuted.

Of course then there's noise. If it's a general rule, but on casual Fridays...

I think part of the problem with this approach is that there are infinitely many hypotheses (due to the infinitely generative nature of language) that are compatible with the behaviour, so to test and refute them all would be impossible. E.g. "only initiate handshakes with your juniors" is compatible with "only initiate handshakes with your juniors who have not been to Mars" etc. This example is silly and may seem trivial, but I think it illustrates the problem of trying to derive a principle or rule simply from observed behaviour.

But isn't that the nature of scientific hypotheses? They are tenuous, only valid until you come across some data that tells you "aha, there's another piece to this rule". The classic would be special relativity vs old Newtonian.

If you have a limited dataset containing the actual generating data, say heights, seniority, age, hair color, day of the week, gender, etc, can't you end up discovering at least the parts that are exposed in the data?

>> But isn't that the nature of scientific hypotheses?

I guess scientists don't guess completely at random though, do they? They use some sort of heuristics, or their instinct (I've no idea what is, no) to decide which hypotheses are worth pursuing.

Most statistical models aren't very good at taking background knowledge like that into account. Not to mention most are also sensitive to "noise" (which is to say, useful information that they can't make use of).

Yep, check out approximate Bayesian computing.

Reply on news.ycombinator.com