Comparing taste in films using pairwise vector comparisons | Hacker News Reader

Comparing taste in films using pairwise vector comparisons

blog.goodfil.ms

37 pointsjohnb16 comments

I'd be interested in reading a more general blog article about their theory behind using "quality" and "rewatchability" as their key user rating. It sounds reasonable at first, but when I think more deeply about it, I wonder what "quality" is supposed to be interpreted as. Is it "how much I enjoyed the first viewing of the film," something more specific like "how skillful was the camera work" or "how good was the acting," or something more meta like "how good I think critics or movie buffs would think the film is?"

I've gone through stages of armchair film criticism, so I've thought about personal ratings a lot. I even drafted a web app to track my viewings and watchlist, and the rating idea I've liked the most is a stupidly simple boolean rating. You could call it almost anything: "Like/Dislike," "Good/Bad," "Enjoyed/Didn't Enjoy," or even something a bit different like "I'm glad I watched it/I wish I hadn't watched it."

http://blog.goodfil.ms/blog/2011/10/07/a-better-way-to-rate-...

Check out this post on the topic: http://blog.goodfil.ms/blog/2011/10/07/a-better-way-to-rate-...

'Quality' is intended to be a more objective score of the craft of the film. Quality of writing, directing, acting; originality of the idea; how influential it is.

'Rewatchability' is where your enjoyment gets factored in. We think it's important to consider 'watching it again' rather than enjoyment first time because it separates out films better. For example, the film Avatar is quite enjoyable the first time round, but IMO not particularly worth rewatching.

Btw, I'm the author of both posts :)

Thanks for the reply. I've thought about this sort of thing a lot. I made a super-basic personal web app a few years ago for recording the movies I've watched and keeping a queue of movies I want to watch. I never added multiuser support or rating, but I gave ratings a lot of thought. My main problem is that a lot of words end up being tied together semantically: "good," "enjoyable," etc. really just end up meaning personal preference. I experimented with the whole gamut of ratings: from the most granular (rating camerawork, acting, humor, effects, etc.) to the least granular ("good" or "not good").

Should "quality" be affected by your film preferences, or is it meant to be an objective (and testable) hypothesis about what some group of people (critics, film bloggers, etc.) would report? If the latter, then I think the quality/rewatchability metric potentially misses information about "first viewing enjoyability," which is what I personally think is much more important than rewatchability.

In the end, I still have trouble with the "quality" metric. If it's meant to be objective, why use user input rather than some aggregation of real data (e.g. Metacritic or Rottentomatoes)? If "quality" is meant to be subjective, then it doesn't seem much different than "first viewing enjoyability," since "enjoyability" and "personal opinion of quality" seem by definition equivalent.

It'd be interesting to see the overall variance on quality vs. rewatachability. That could give some clue on the objectivebess of users' answers to quality. Perhaps quality is just one of those things that is more universally objective than it is subjective?

Another thought is that there may be some ratio between quality and rewatability that signifies the first-time (but not repeated) watchability of a film.

For example, 5 stars for quality and 0 for rewatachability doesn't tell me I should watch the film if I haven't already. But maybe 2 stars for rewatability/subjective enjoyment is enough justification to watch it to appreciate the quality?

I agree with these definitions. For some reason, most of the films I really like to watch over and over again are actually not the films I'd say are the highest in quality.

e.g. I'd consider The Shawshank Redemption to be a high quality movie, but I have no particular desire to watch it again (even though I enjoyed it). On the other hand I can't watch enough of Big Trouble in Little China even if it's a cheesy movie.

Some of my favorites though have both qualities, e.g. Blade Runner.

How is this better than a normalized cosine similarity? The vector being arbitrary, but in this case being a normalized value on quality and rewatchability.

Cosine similarity would also let you express pairwise similarity as a single normalized value, instead of a 9-way comparison.

Interesting read. Vector Victor sounds like a linear correlation algorithm. I wonder what coefficient they're using under the hood...

It's not actually linear correlation, since we effectively normalise the pairwise scores to [-1,0,1],[-1,0,1] (nine possible combos). We're exploring blending in a few other signals along the way, but we wanted to see how far we could get by discretising the pairwise comparisons in this way.

Once we've collapsed all pairs down to a Vector Victor, we treat matching Vector Victors as a thumbs up and non-matching as a thumbs down, take the square root of both then take the lower bound of the Wilson interval as our ranking function.