Hacker News Reader

Comment by daniel-levin | Hacker News Reader

Speaking of ambitious ideas and 'middlebrow dismissal', I'd like to see the algorithmic dismissal detector that PG wrote about. I'm sure the denizens of HN know a thing or two about computational linguistics. Is this sort of thing (algorithmic detection of middlebrow dismissals) possible?

View on HN

  > Is this sort of thing (algorithmic detection of
  > middlebrow dismissals) possible?

I'll give you a middlebrow dismissal: No. Definitely not at the current state of the art. In fact, not many people agree on whether a given comment qualifies as a "middlebrow" dismissal; personally, I consider many of the examples that pg has labeled as such pertinent points.

If in doubt, I prefer a good middlebrow dismissal to the shallow cheering of the overly excitable.

What is 'the current state of the art'? Note that I am a total dilettante in the area of machine learning/NLP.

And why couldn't the top k HN users (patio11 et al) be given the option to classify comments {dismissive; troll; shallow-cheering; ...;} and this human classification could be the training set of a supervised learning system that could use NLP algorithms to derive useful metrics. Simplistic example: dismissive word ('sucks'; 'never work', 'impossible') density correlating to something that has been flagged as dismissive. Or is this an impossibility? Again, I do not have the hands on knowledge to be able to call this one.

The immediate flaw in that approach is that "patio11 et al" are top contributers because they post a lot and have valuable opinions - NOT because they are always right. They're not (I'm confident they would agree here). Therefore their opinions would be inappropriate as an AI training set.

That's true. But doesn't that kind of sidestep the question?

It's always going to be a matter of opinion what constitutes middlebrow-dismissalness, so if we accept that then the question is really:

"Is it possible to automatically determine if this post would be judged in this way by this person (or group of people)?"

It seems like that's a question that has at least a possibility of a "yes" answer.

Sure, totally agree, but a consensus by verified posters was more of what I was aiming for

I guess I am going to pitch my potential solution again:

http://gkosev.blogspot.com/2012/08/fixing-hacker-news-mathem...

it would probably have more weight when i release the test code and datasets though...

Not an NLP guy by any means, but aren't we at the point where we can at least guess sentiment algorithmically? Couldn't you combine that with the commenter's history, the length of the post and the presence of charged words such as "nonsense" and "silly?"

Ternary sentiment detection rates are currently around 30-40% for Spanish. English rates might be better, but not by a lot. That means you give a piece of text to your classifier and it will tell you correctly if has positive, negative or neutral connotation at most 40% of the time.

In short, we are still worse than a fair coin toss at sentiment detection.

Language is not tricky at all, no sir. Can you see why? :-)

Pardon my nitpicking, but wouldn't the metaphorical "coin toss" in a case where there are three possibilities be 33%, so 40% is very (very) slightly better?

For cases with only two outcomes, you can never be worse than a coin toss. If you're achieving 40% accuracy in that case, all you have to do is invert the output of your algorithm and suddenly you're achieving 60% accuracy.

Not a terribly important point I'm making, I know.

Reply on news.ycombinator.com