Comment by haidut | Hacker News Reader

Comment by haidut | Hacker News Reader

Parent Full thread

We've nee collecting top rated news from Yahoo News, NYT, Washington Post, etc for the last 3 years. Almost all major sources have sections "Most viewed", "most read", "most email" etc. So as we have been collecting the top ranked news we also kept track of topics that have been top ranked at multiple sources. For instance, if a news article on Iran's riot gets to the top ranked in both NYT and Yahoo News it gets more points in our training set.

As far as your other point, Misleading is really a less legally loaded word than "deceptive". You are right, it's not very clear why the articles are misleading but the bottom line is that the language used in the article has high "deception markers" that other articles from the day. So when you sort by Misleading, it's not really that the article is beyond a doubt deceptive, it's just the ones with higher rate of deception markers (i.e. content and structural indicators associated with deception). The science behind is pretty solid and comes from forensic psychiatry - i.e. interviews/interrogations with criminals and analyzing their statements for deception hints. So for alack of a better term Euraeka essentially implements a linguistic polygraph. Thanks for the other comments, we'll work on fixing the issues.

Are you conflating/clustering articles? ie what constitutes the "same story" in your system?

Those "most viewed", etc boxes are often placed by editors, not by impartial algorithms. How do you control for that?

Yes, we definitely have clustering. In fact, it's more extensive than simple word distances like cosine b/c we also take into account synonyms and word relationships (set membership, etc). For instance, in our system sentences like "Tiger chases antilope down the river" is very closely "related" to the sentence "Lion is pursuing a buffalo by the lake" b/c both sentences essentially say that a large cat is pursuing a prey of bovine origin near a water source. In terms of the "most viewed" and how we control for that - like I said we cross-track news on multiple news sites and weight the cross posted one more often. We also cross-validated the most important tags for an article by using Google Trends data. Basically we tracked topics on multiple sites and then performed some statistical analysis to see how those topics did over time based on their presence on the web (topic momentum and longevity). We also run a partial search engine in house that crawls a subset of the web so we can ensure that the numbers we get from Google/Yahoo are legit. Finally, there is linguistic theory of topic popularity and how memes propagate over time. We use some of that theory to control for the crowd effect - i.e. sometimes people pick up and spread topics that are of no real importance to the world. Example: Paris Hilton's latest escapades may be widely discussed online and appear important news but the latest report on the recession estimates and projections is of much higher "impact" to society. So we try to account/estimate some of that "impact". Combining all factors gives an article a composite score. No two article really have the same score but a lot of articles cluster close to each other in terms of their "importance" cores. We fed the articles in a machine learning algorithm that is a combination of Support Vector Machine, Neural Network, and Naive Bayes and when a new article is fetched by our crawler the model "preditc" its various scores (controversial, engaging, popular) based on the data set that it has already learned. Deception detection is much trickier and is almost entirely analysis based - i.e. no machine learning there. There is quite a bit of research on deception detection published online. Just search google for "deception detection ext:pdf" and it will come back with a lot of results.

Reply on news.ycombinator.com