Mozilla Is Crowdsourcing Research into YouTube Recommendations
foundation.mozilla.org
foundation.mozilla.org
Now the topics it recommends to me are non-political, but I can very well see that if you fall down one of more extremist rabbit holes thats all you will be fed on your recommendation page for months.
According to the current scuttlebutt lore, Youtube makes a concerted effort to steer users towards authoritative sources (typically the big media brands), especially on spicier subjects. This is in response to the recent slew of articles accusing Youtube of user radicalization.
I think Youtube's recommendation algorithm is an abomination that has caused real harm to society.
Try watching a comedian that's slightly edgy like bill Burr, you get immediately showered in videos of people "showing feminists what's what" or "shutting up liberals". I'm not even American, not right wing, but YouTube seems to try its hardest anyway.
Googling around shows there used to be a 'Don't recommend this channel' button which would be awesome to have back.
I find it very annoying listening to a two hour debate, understanding difficult topics and then disagreeing with the speaker and being unable to block content from said speaker.
There's a bunch of one-topic people pushing their opinions out to the youtube user base and it would be great not to have to see their face in 30 different suggestions after you find yourself deeply disagreeing with their opinions or you're just done with their conversations after hearing all of their views.
It's still there on my Chrome desktop or Youtube app on android.
And yeah even within music videos, the recommendations seem really basic. Compared to Spotify, Google is looking like real amateurs when it comes to recommendations.
But yeah, the music cycle is funny. And for some reason, no matter how many times I say "please don't show me this video", YouTube really wants me to listen to Mandolin Orange. I can't escape that recommendation even though I think I listened to them once and decided it wasn't to my taste and have always removed them from the queue. (Sorry, I'm sure they're nice but it's just not for me)
I agree about Google bad recommendations, but for me Spotify it at that same bad place too. It used to be way better, but now 80% of my daily recommended playlists are stuff that I have in my own music library, just shuffle played. It's absolutely terrible.
...
Why is this so bad? What are your hundreds of data scientists doing?
With enough time Tiktok will look as bad. I.e. I don't think it's algorithm related "for the most part"
I'd love for it to learn that anything with a shocked face thumbnail and CrAzY question for video name is not desired.
That's the problem with marking things as not-interesting. The algorithm will wildly overreact in the opposite direction.
The only thing that works for me is finding the video in my watch history that's causing the recommendations and removing it there. Often it's even something I didn't watch all the way through or even gave a thumbs-down; apparently thumbs-down also means I secretly want more of this content?
YouTube's recommendation algorithm is somehow worse than useless.
It's like a wild species with vibrant, sometimes garrish colors saying 'Look at me!', where to discriminant viewers it's typically poisonous.
I'm pretty cautious.
But for a large number of my favorite content creators it's a bait and switch. Bait your clicks and actually provide really meaty content.
That aside, my personal favorite trend in the digital media space are the 1-4 minute tutorials for people who already know the software.
These people typically make money outside of Youtube and don't rely on snaring views.
> "binging random content"
gets recommended random popular content
> "why are recommendations so bad"
They can't read your mind, you know
I'd take your point if they only recommended bass videos, but presumably it's just one recommendation in a list of many, with the point being to explore your preferences while also satisfying your demands
davie504 is basically a musically-themed comedy act, so this doesn't seem that unreasonable.
Other than that, I'm pretty happy with my recommends. I think the Google spooks have built a good file on me.
https://addons.mozilla.org/en-US/firefox/addon/youtube-clean...
Say you go on reddit/r/iloveredcars - all you will see upvoted (ie ... RECOMMENDED by users) will be red cars. You'll never see blue cars. You'll never know they even exist. All you've ever seen are red cars. Not only this but you'll notice that over time it's always the same red car pictures being reposted over and over again.
By inserting "wrong" results a small percentage of the time this ensures that users will be exposed a tiny bit to different content, that may be they end up liking, opening an entiere new tree of recommendation opportunities.
Now I'm sure some people don't want this and they do want to see the same red car pictures over and over, but personally, I appreciate that they do this.
Slap like right now
The videos that YT recommends over and over again are probably videos that have a high ratio of advertising revenue (meaning good demographics) to playback cost (meaning low overall byte size) and a high likelihood of going viral by being shared off-platform (meaning broad appeal, non-controversial, short, safe for religious conservatives, safe for work, and easy to summarize within 280 characters).
The videos you want to watch don't meet these criteria, so Youtube won't find them for you.
I'm right now on a browser without my account and on which I don't usually watch YT, basically only when someone pastes a link while gaming. About half of the front page are things I might have clicked on when bored. A third are long. Two are definitely controversial. I don't buy your hypothesis.
Disclosure: I work in Google, but my closest contact with YouTube was drinking whisky with a YouTube SRE some four years ago.
YouTube only knows your interests by the videos you watch. It knows you like speedruns, so it will show you speedruns, easy. But you certainly have other unrelated interests: cooking, woodworking, old cars, whatever. So, from time to time, Google takes a chance and shows you a random topic just to gauge your interest.
It works on a global scale too. They want some variety, they don't want to depend only on a few topics and previously successful YouTubers. So sometimes, they may pick a random video and so it to everyone, and if it succeeds, they have something new.
It is actually rather clear when you look at your recommendations to determine which are which. Typically you are going to find a mix of videos related to the video you just watched, videos relative to your global interests, trending videos, and random stuff.
If some generally effective machine learning algorithm acts really stupid from time to time, chances are that it is exploring, that is: it is asking you a question instead of giving you an answer.
Edit: someone mentionned the multi-armed bandit, yes, this is such an algorithm
This is not the same algorithm that they've always used. It began changing somewhere around 3 or 4 years ago and it's completely different now. Honestly, I think they've intentionally broken it. I used to derive a lot of enjoyment from going on an algorithm enhanced "wikiwalk" of YouTube content, discovering many interesting and bizarre videos. It's impossible to do that now, you just wind up stuck on a channel or going around in circles.
To put it plainly, I think YouTube has decided to bury all of the fringe and weird one off content in favour of mainstream channels because they can't monetize the former. Good job Google, you killed another golden goose.
control + alt + m
notify-send "mpv loading $(xclip -o)"; mpv --player-operation-mode=pseudo-gui "$(xclip -o)" || notify-send 'mpv error'
Could you tell me how ? Would love to have that feature. They seem to have turned this off.
This applies to YouTube music as well. I find this incredibly annoying.
Know what's even worse? Searching for something, then liking that song. On my end, that generates an incessant stream of titles from the same artist that no amount of downvotes can stop.
YM sees the pattern +--------------------- and yet continues to predict +.
Youtube recommendations are best effort. That effort has a computational budget and a time limit. The fastest and least computationally intensive recommendation is to pull the last set of recommendations out of the on-device local cache.
The next fastest thing is to recommend whatever is cached near the edge of your internet connection whether it fits your previous viewing or not. It generally does not do this. Instead it recommends edge-cached material with cosign similarity to your preferences. However, there might not be much of that if your preference is for outlying content.
The slowest and most computationally intensive recommendation would be to compute a new tensor of your preferences and run cosign similarity across the entire vector space of Youtube. It doesn't do this for obvious reasons even though that's what we all want.
Of course, this ignores the issue of Google revenue. Advertisers targeted you, target you based on the videos you've watched. So there's that too. Google recommends videos for which it has relevant advertisers.
Google does not have that much interest in serving new, interesting videos over what earns the most clicks and therefore advertising. A lot of users might also be pretty fine with what is already served. Although I have seen more and more alternatives popping up, several with a monthly payment plan instead. While free content is nice, a paid plan incites different business practices than one where revenue is closely coupled with the number of clicks.
I think this is exactly the problem.
Contrast this with the recommendations I get from my paid spotify account where I don't get ads: It's absolutely Excellent. Seriously, whoever worked on Spotify's recommender system algo deserves a huge pat on the back. They have the best recommender I've encountered anywhere on the net, and I pay a lot of attention to recommenders. It's just light years ahead of everyone else.
I think part of it is them removing certain content from the recommendations. I would be happy with a chronological order of all videos from all my subscriptions. Perhaps with groups of subscriptions, so I can make a group of comedians, a group of nerds, a group of girls dancing, etc.
The real problem is that youtube is still so much better than everyone else at just playing videos. How has it been 10 years and there is still no real competition in the realm of usability. Even most paid services are not as good. I think a bunch of the streaming services should pool their resources and come up with a good open source player with libraries for all the major frontends and backends. Instead of all of them poorly reinventing the wheel over and over.
Video is much much more complex than a .mp4 file on a server - the effort required to get the first frame from the server to the users screen as quickly as possible, and then not drop any frames after that, is pretty high to begin with, but it's even higher if you don't have control of the exact way the browser allocates CPU/GPU time for example...
Is this not the subscription view? I've never noticed a video from any channel I subscribe to that didn't show up there.
There isn't any more "depth" to recommendations anymore. Right now it's the same recommendations over and over again. Lots of celebrity, corporate media, etc.
When youtube wanted to grow, their recommendation was so good. You could go down that rabbit hole forever really and actually discover content/creators. Now that they have a monopoly position, it's about limiting options and guiding you to established content.
I don't understand how the recommendation algorithm seems so brittle, given that advertising is their core business, and that I am manually doing so much of the work by subscribing to channels I enjoy. A simple feed of recent updates from channels I subscribe to would be an improvement; in fact I think I mostly find videos to watch from Twitter.
My often used example is the time I went car shopping with my sister. She was buying a new car I went with her (age old women vs salesman bias). The dealership was over an hour from my town, my sister lives in a different town. I didn't do anything other than stand there, no info of mine given, no forms filled out.
Back home two hours later YouTube has showing on recommendations for me videos for new car shopping.
Youtube knows exactly which videos I've watched and which I haven't.
Why does it think it's a good idea to ever recommend videos that I've already watched?
- fitness videos
- music
- how-to & instructional videos
On mobile YT suggested me to watch an old video from a channel I'm subscribed to. It's a good video but since I have watched it already (with my account logged it) and remember it who cares about it. This one time the interface asked me if it's a good suggestion! So I clicked the bad suggestion link, and then the app asked me to categorise why is it a bad suggestion. All the options were about the video itself. The categories were "overly sexualised", "gory", "boring" etc. There was no option to say "I have seen it already you silly". Way to set up a feedback system to get no useful information out of it.
At least half of the time on Youtube, I watch classical music, and you would think Youtube would recommend me some classical performances that I haven't yet seen.
Nope. It keeps feeding me exactly the same videos I've already watched, despite being basically the largest catalogue of classical music videos.
What's the point of a recommendation system if it doesn't even recommend new video of the same category I like? Might as well just shuffle my watching history.
But maybe their recommendation engine is hyper-focused on the matrix-completion framework and doesn't use any information other than which users watched which videos.
For a company that has so much of my data and indexes the entire internet, it sure is surprising how bad they are at recommendations.
(Although I only take recommendations from the front page, and ignore the playlists entirely.)
Because YouTube doesn't care what you want--YouTube cares what it can monetize.
I suspect that YouTube gets very little money from showing you classical performances but gets lots from showing you the latest "Funny Cat Video".
You are the product, not the customer.
Once you internalize that, stuff that Google does makes sense.
To be honest, the more I think about it, the more I don’t know what money is YouTube trying to earn.
It’s probably just a dumb algorithm.
I think I heard somewhere that a lot of kid videos rank in the top stratosphere of view counts on YouTube because kids just play stuff on repeat. There's probably a lot of repeat plays going on across YouTube.
> You are the product, not the customer.
> Once you internalize that, stuff that Google does makes sense.
They published their algorithm. They rank items according to predicted watch-time. The other poster is probably correct, in that many people use these long classical music videos as background music over and over (i.e. high watch time) and so they're highly ranked.
Covington, Paul, Jay Adams, and Emre Sargin. "Deep neural networks for youtube recommendations." Proceedings of the 10th ACM conference on recommender systems. 2016.
Nvidia furthermore implemented/released it for TensorFlow, and for their new Recsys engine:
https://ngc.nvidia.com/catalog/resources/nvidia:wideanddeep_...
In the same way that online advertising needs to display millions of ads to reach only hundreds of people, the recommendation system may not be intended to work at 100% efficiency.
YouTube "recommendations" work as far as I can tell. If you take two people searching for some subject area on YouTube, and only one of them knows how to search past the initial "popular" recommendations, each person will produce very different results.
If neither person is a skilled searcher (most people are not), they both end up with the same results. Those videos receive thus more and more views. Videos with millions of views help Google sell online ad services. If views were distributed more evenly across all videos, would that make advertisers more interested or less interested.
What's the point of a recommendation system that does not recommend a new video in the same category. To promote certain videos to attract large audiences and thus draw more interest from potential advertisers. More advertisers bidding on fewer popular videos would seem to favour higher prices paid to Google.
Some of the best videos I have seen on YouTube have less than 500 views, some even in the single digits. I found them through automated searches, not "recommendations".
Google makes information accessible with the belief that relevance is directly proportional to popularity. The company was founded on this idea.
The research in question was limited by not using logged-in accounts with a significant view history, but it's still the best attempt to quantify the YouTube algorithm's influence that I've seen so far. Mozilla's initiative sounds interesting, but the fact that they're asking users to only report negative interactions rather than getting a representative snapshot of all YouTube use makes me suspect that Mozilla is approaching this from the perspective of advocacy, not scientific inquiry.
--- [0] https://arxiv.org/pdf/1912.11211.pdf [1] https://recfluence.net/
That means that when I want to leave a website open on my server for a month, I use chrome because I know exactly how much memory it's going to use. If it was firefox, at the end of the month even the swap would be filled.
I don't know if you count memory exhaustion as a 'crash' per se.
This is a well-known problem. What's hard to believe?
Yeah, like paying three million to its CEO annually. That's not going into Firefox development alright.
As far as I know, Mitchell Baker is still Mozilla's CEO. And even if she resigned, maintaining her salary would have been at odds with keeping many employees from being fired.
>>I criticize unreasonably high salaries on a nonprofit organization that just laid off almost three hundred people
>"you are saying that they can build and manage a development team without executives"
If you want to engage in a discussion or a debate, at least be honest about it.
Mozilla is a nonprofit, not a charity. Their executives already receive less than market rate for similar companies, by some accounts far less. They can't offer nothing.
--
[2] https://github.com/miscavage/De-Mainstream-YouTube-Extension...
That said I don't think it's what Mozilla and many commenters here are looking for, they're looking for banning content they disagree with (i.e. censorship).
That's an extremely unfair read of the situation. The extension doesn't even ban anything. It is a research project attempting to understand why the YouTube algorithm recommends the videos it does. Whether or not you think Plandemic (a nonsense conspiracy theory video) should be banned or not you can still see research into the YouTube algorithm as a worthwhile endeavour.
This is not just an open-ended inquiry of what youtube is recommending. It's about finding the bad and getting rid of it.
EDIT: I also want to remind people about how Mozilla used its push notifications to call for a Facebook boycott over objectionable content.
What part of this is untrue?
And again, you're saying "getting rid of it" without evidence. It's a study. It does not mention banning anything, anywhere. If the argument is that they should not study this because it might lead to bans in the future then you're being pro-censorship in order to promote anti-censorship which doesn't make a whole lot of sense.
I'm not saying they should not study this thing. I am just discussing their aims, because I think it's interesting, and it may inform others' choice of whether to participate or not.
The characterization of pandemic conspiracies and political disinformation as harmful.
It's like kids and allergies. You never hear about kids who grew up on farms being allergic to animals - it's always those whose parents didn't have any when they were growing up.
If people aren't subject to misinformation, they'll never develop the sense of who's lying and who isn't.
It used to be that we gave common-sense advice - "don't believe everything you read on the Internet". Now, it's the other way around - "we must cleanse the Internet of harmful content".
Being exposed to misinformation is good for you, and it's good for democracy.
Meanwhile, the actual state of affairs is[0] that there's one allergic farmer child per three allergic non-farmer children. Don't believe everything you read online.
They seem to have stopped this and now the recommendations featured on my homepage are all over the place.
Interestingly, I think there are a couple of variants for the number of videos to display on the home page, too.
Try refreshing the home page until it shows up, maybe?
1000s of hours of videos are uploaded every minute. 5+ billion videos are watched daily. Are you not rather quite pleased that YouTube can deliver highly relevant top-20 recommendations within milliseconds from such a large and changing corpus, while keeping track of your past preferences, your interactions, your social circle's preferences, popularity and trends, content features, etc?
It's pretty amazing to me. I think people expect recommender systems to read minds.
That's the way it used to work, now it adds that new category to the list of categories it thinks you like (top of the home page), and you can filter or include that category.
I'm subscribed to all this stuff but it never seems to give me similar content, it recommends all the videos I've already watched.
If I happen to watch something new, like a People's Court segment, then all I see is People's Court stuff.
There never seems to be balance.
Then there's the problem where a new video (like Gourmet Makes from BA) gets posted on a channel I like. I'll watch it with my partner on his machine, but then YouTube just chokes up, wondering why I never click on it, and then shoves it in my face ad nauseum.
Like, their search engine doesn't even seem to scratch the surface of content they're holding. It seems to be really stupid machine learning or AI.
Why can't it show me balanced recommendations from everything I watch or subscribe to, toss in some new things which are similar/popular, and maybe differentiate between content likely to be consumed multiple times (music videos) vs stuff people don't often watch a rerun of?
They seem to have so much data they could work with. They have amazing engineers and Google's expertise in algorithms.
Facebook even had problems with news feed being overrun with low quality content early on but seems to have figured it our fairly well- at least in my experience on the site (I know there are big echo chamber problems over there as well, don't get me wrong).
Even a lot of my non-techie friends seem to complain often about how terrible YouTube's recommendations are.
They are really not. I struggle pages after pages to find something interesting. And then days later i stumble on a link that has interesting stuff randomly. Even search is bad. And there's always the knowledge that google is censoring stuff and manipulating people.
There's definitely space for better youtube/video recommendation site.
Mozilla Foundation, in contrast, makes and spends relatively little compared to that. Seriously how much do you think this crowd-sourced project is costing the Foundation?
https://www.propublica.org/article/facebook-blocks-ad-transp...
I don’t see any other likely outcome of this “research” other than a fresh new media hue and cry detailing chains of “wrong-think” that will lead to activists who masquerade as journalists calling for more content they deem unfit for the plebs consumption.
Same as politics, you want a maximum of people to be convinced by your idea/opinion/etc to make it happen faster. If you build your idea it might be great but if no one is convinced of it it won't work.
youtube.com##.ytp-suggestions
youtube.com##.ytp-pause-overlay
youtube.com##.videowall-endscreen
youtube.com###related
youtube.com##ytd-browse[page-subtype="home"]