Hacker News Reader

Comment by moxious | Hacker News Reader

This is a very annoying part of non-explanatory models. I think the result defies common sense a bit, and the model can't explain why this is so.

So in the circumstance, why should we believe it's generalizable?

View on HN

This also goes to the heart of the problem with deep learning on neural nets. We have this algorithm that apparently identifies homo and heterosexual people, presumably based on a variety of subtle features, but we have pretty much no clue as to which features and why.

The human judges may have been less accurate, but they could likely explain each decision they made and the visual features they based their decision on.

Humans are known to be unreliable in explaining how they come to conclusions as well. Humans just like to pretend they can verbalise all knowledge ;)

Even if they verbalized their knowledge incorrectly they give you something, which if you chose, you could further test / replicate. In other words even if they're BSing they're still falsifiable, not so "magic models" when their publisher may not want them falsified

Some are better at it than others, and no doubt many are pretty bad at it, but I have yet to see a neural net explain to me, accurately or not, why it came to the decision it did.

Don't forget that we can listen to their attempt at verbalizing that knowledge and then, in turn, draw/verbalize our own sketchy conclusions....and so on.

Reply on news.ycombinator.com