sadikkapadia1

That is what I have been told. It is however clearly messed up.

Surprising good.

It is hard to sell technology to companies when they have their own teams (often using free libraries). Embedded teams are always experts and will often discredit better technology. Often the only method of testing is A/B. These can easily be manipulated. For instance at Netflix (ignoring more blatant practices), P-hacking (run thousands of simulations and report ones that worked), and HARKing (come up with a hypothesis after the results are known) are rampant. That is part of the reason the recommender has been degrading over the years.

This is what working in a big company is like sometimes. Imagine a person like MaxLeiter working in Netflix. You would think that all staff would be happy that their work became easier. But a minority seem to have a zero-sum mindset.

When I finish my current work I will talk about that also. Some of my older work is speech recognition. Download my thesis.

I don't keep track of time. That system is old technology. I did confirm from a Netflix employee that they still use it a few months ago. Deep learning, LDA (even one of Xavier's pet projects - k-means), did not do better.

I wrote the recommendation system at Netflix (still in use after 5 years). Primary problem was company politics. Many groups were not happy that one person could write a system that was better in A/B test, had more uptime and cheaper to run. All of it (ML, production, monitoring), was custom code.

The future is a long time. Eventually faster computation, larger memory would allow taking smaller and smaller steps during training (coupled with avoiding "bad optima" with stochastic training). All of this would improve robustness of training.

The domain dictates whether degree of opacity (or other attributes), would rule out deep learning.

Netflix recommender does not use deep learning (which is pretty amazing given how badly they have messed it up). From the conversations I had (a couple of years ago), they gave up with it. I'm sure the Youtube team could do a better job on the Netflix data then they managed to do.

https://research.google.com/pubs/pub45530.html is the most complete recent paper I've seen.

I wrote the recommender at Netflix about 5 years ago (every line of code). Netflix has been degrading it since then. The problem is that many companies are hotbeds of politics over expertise. Recommender, UI, A/B tests, etc are an excellent venue for politics at the expense of the product.

Some anecdotes from Netflix: https://www.reddit.com/r/MachineLearning/comments/6xiwr4/d_w... (Scroll down for my comments. Ignore namp243 comments - see below).

Another example is the "Netflix prize team" at Verizon/Yahoo. They refuse to share data with any other groups (they are afraid of being discovered), leaving other groups literally nothing to do.

In my opinion Youtube recommendations are improving. They are still pretty bad but they are trying interesting approaches.

There are many intelligent replies here.

5 years ago I wrote the recommendation system that Netflix uses (and has degraded since then). One major problem is in the past certain senior Netflix managers are only interested in self promotion (I would hate to extrapolate to the current ones - even if the extrapolation is reasonable). A/B tests are a perfect device for this. What is a better recommender? They were not interested in improving the product. It is easier to win by politics/lying/obfuscation/omission/plagiarism then come up with better ideas. If the company goes down they move on with a good resume and the games they played (for instance USPTO fraud), are hidden.

Scroll down for my comments here: https://www.reddit.com/r/MachineLearning/comments/6xiwr4/d_w...

Some companies are like this. For instance the "Netflix prize team" at Verizon/Yahoo refuse to share recommendations data with other teams leaving them nothing to do. They work in a bubble and will actively try and remove anyone who might be a competitor.

It's sad that Netflix decided to pursue this work (+ other even more brain-dead projects like rewriting the command-line parser, "switching to OO" - examples of Xavier's initiatives). They could of 5 years ago pushed the beta system that has 40% extra performance. (I'm currently at a factor of 3 better performance).

Netflix is no different to any other company. Your happiness depends on your boss. And even single teams go through bad patches.

Some background. I wrote Netflix's recommendation system. Netflix later went on to take part in fraud against USPTO which now led to Netflix with no ownership of it's "most important technology". Xavier Amatrain https://www.linkedin.com/in/xamatriain/ was my boss. Xavier went on to hire lots of staff to try and replace my system and was unable to do so. In the end he started claiming credit for systems I had written without understanding the technology. He still presents his incorrect knowledge in public conferences. The result is that for 5 years Netflix has been running my alpha system without improving the technology (various techniques that Xavier tried and k-means, LDA and deep-learning). Netflix never did push a version I had written with 40% greater "click-through" rate, 3 times greater speed and order of magnitude less memory.

One example conversation I had with Xavier after he left Netflix (para-phrasing of course): Me: How is my system doing? Xavier: We don't use it anymore. We stopped using your system 1 week after you left. Me: Really what do you use? Xavier: I cannot tell you it is secret. Me: Who wrote it? Xavier: You don't know him. We hired him after you and he wrote and pushed the new algorithm. Me: So that new person, wrote a new recommendation system; pushed it into A/B test; got test results and then pushed it into production in 1 week. Xavier: Yes - he is better than you. All was this is a lie (and not out of character), because I have been in touch with other engineers post-Netflix and in fact I've talked with Justin on a number of occasions on the phone about how to keep my system running. Later Xavier invited me to work for him at Quora. He offered in his own words "an unbelievable salary". I of course refused. I met Xavier sometime later and tried to talk to him about a particular bad presentation of his. He got upset and shouted "Fuck Off, I'm not interested in listening to anything you have to say" multiple times. We were on the Facebook campus at the time. He left at that point.

This is just the tip of the iceberg. Always remember https://www.goodreads.com/quotes/65213-briefly-stated-the-ge...