Recommender Systems for Social Bookmarking (2009 Ph.D. thesis) | Hacker News Reader

Recommender Systems for Social Bookmarking (2009 Ph.D. thesis)

ilk.uvt.nl

37 pointsjashmenn8 comments

I was just discussing about recommendation engines with a friend yesterday.

My pet peeve is that i find a lot of the companies,datasets,research focus on the positive aspects of recommendation. There is probably a lot more negative data we're missing out on, which is not being collected a lot. That data consists of things i would "hate". I would want to chart out a hate pattern of the internet and an anti-pagerank of sorts.

I was reading a post about the same thing yesterday(1), linked along with some questions of my own at http://blog.sidmitra.com/where-is-my-dislike-button.

Ref: http://blog.superfeedr.com/social/algorithm/a-social-filteri...

Good point. Beyond the usual 'people who like this also like', there are some interesting inversions:

  1) People who hate this also hate
  2) People who hate this also like
  3) People who like this also hate

Judicious use of these might help to provide the serendipity you mention in the linked post.

But in practice, I think there are a number of reasons that you don't see 'dislike' buttons as frequently.

  1) The common KNN item-item approach doesn't handle negative correlations. 
  2) Without explicit ratings, dislike is harder to infer than like.
  3) Sites just don't like appearing to be negative.

1) Space for algorithmic innovation? (which is the coolest thing to be in front of but I guess there must be already some algorithm application to handle this case) 2) Why? Could you give some examples? I ask because I also think that rating-less like is dependent on the subjects personality. For me a dislike button should be much more meaningful because as a optimistic there is a bigger range of things I like with a bigger variation of likeness, while when I (would) hit dislike I must be pretty sure that I don't like it. 3) I agree but the implication is that sites should use negative qualifications differently, it shouldn't be shared with others in the same way the likeness is shared unless, as in your suggestions, they enable you to find better content. I don't go telling people I dislike that I dislike them, I just avoid them. The approach should be similar.

Personally, I think there is a lot of room for algorithmic improvement on recommendations. Most of the existing systems are really prediction algorithms, which have been shoe-horned into making recommendations. I think this is primarily because accuracy of prediction is easier to measure (RMSE) than accuracy of recommendation, and the academic papers need something they can measure.

I agree that inferring 'like' is also difficult, and definitely prefer explicit rating systems. But for a site like Amazon, it's often a reasonable presumption that a user who buys an item likes it. But inferring that they dislike all the items they don't purchase is much less reasonable. Same for viewing webpages, or listening to music, or renting movies. If you want to be sure of a 'dislike', it probably will require action from the user.

  1) The common KNN item-item approach doesn't handle negative correlations.

Er, why not?

I may have been too aggressive trying to shorten that statement down to one line. What I meant was that item-item approaches, when used as recommenders, won't make recommendations of the 'users who hated X also liked Y' variety, and instead limit their recommendations to items that correlate positively with the user's favorites.

The usual approach for item-item is to precalculate similarities for all items in the set, using Pearsons correlation or cosine. There's no problem with negative correlations being used to make predictions, the problem comes when you try figure out what to recommend.

The recommendation step consists of looking at the items a user has already rated highly, and then looking up which items have a strong positive correlation to these, then determining which of these to put at the top of the list. Items with negative correlations will thus never be offered.

There's nothing to prevent one from throwing in anti-correlations of hated movies, but I've never heard of it being done. There's also nothing wrong with making predictions for every movie in the set and then choosing the highest (taking full account of negative correlations) but this is generally thought to be too computationally expensive.

The main issue I had building or of these for delicious was not the algorithm (many techniques yielded pleasing results) but instead scaling to get any reasonable performance.

This obsession with recommendation systems! the thing is when you like a lot of things that algorithms can't tell (or better: there is not enough information to train those algorithms).

Reply on news.ycombinator.com