Comment by sadikkapadia1 | Hacker News Reader

Comment by sadikkapadia1 | Hacker News Reader

Parent Full thread

I wrote the recommender at Netflix about 5 years ago (every line of code). Netflix has been degrading it since then. The problem is that many companies are hotbeds of politics over expertise. Recommender, UI, A/B tests, etc are an excellent venue for politics at the expense of the product.

Some anecdotes from Netflix: https://www.reddit.com/r/MachineLearning/comments/6xiwr4/d_w... (Scroll down for my comments. Ignore namp243 comments - see below).

Another example is the "Netflix prize team" at Verizon/Yahoo. They refuse to share data with any other groups (they are afraid of being discovered), leaving other groups literally nothing to do.

In my opinion Youtube recommendations are improving. They are still pretty bad but they are trying interesting approaches.

Can you share any ideas on how you would implement a new recommender these days? Any papers you like? I am trying to learn more about this topic, but good information is hard to come by. The best resources I found so far are presentations from Netflix and Spotify, but they skip over so many details and assume so much knowledge that it's hard to get good results without being able to consult someone with experience.

https://research.google.com/pubs/pub45530.html is the most complete recent paper I've seen.

Thanks! So, would you say the future for recommendation is deep learning? While I am not opposed to it, I find it very opaque.

The future is a long time. Eventually faster computation, larger memory would allow taking smaller and smaller steps during training (coupled with avoiding "bad optima" with stochastic training). All of this would improve robustness of training.

The domain dictates whether degree of opacity (or other attributes), would rule out deep learning.

Netflix recommender does not use deep learning (which is pretty amazing given how badly they have messed it up). From the conversations I had (a couple of years ago), they gave up with it. I'm sure the Youtube team could do a better job on the Netflix data then they managed to do.

Reply on news.ycombinator.com