Using Posters to Recommend Anime and Mangas in a Cold-Start Scenario

yes, recommendation systems are a very vibrant branch of machine learning.

I don't understand the paper fully but the general idea seems to be, if an item like an anime don't have enough ratings from users for standard techniques to apply in analyzing it for recommendations, using metadata derived from its poster (e.g. a girl holding a sword) can help.

(skimmed paper)

The classic recommendation problem is the following: given a user and the items (mangas) that they like, out of some universe of items (mangas), how can we recommend new items (mangas) that they are also likely to enjoy? Typically this is done via collaborative filtering or some other method, i.e. people who like the same mangas as the original user also enjoy other mangas, so we recommend these to the original user.

A very common problem occurs when you have a new or obscure manga, aka the cold start problem. There are no reviews to use when finding similar mangas (which is our input to the recommendation system). The authors propose extracting visual information from the posters of these less commonly reviewed mangas that finds characteristics of the manga. The theory is that users that like mangas with 'girl with sword' will also like other mangas that have 'girl with sword' or perhaps 'girl with bow' but probably not 'girl with book' (I made these tags up).

Most content-recommendation systems you interact with are collaborative; with data about what users like, they use "likes" from one user as "recommendations" for another user, when those users generally like the same things. So if I like content A, B, C, and you like content A, B, D, they might recommend C to you and D to me.

This approach is (comparatively) easy to build, since your recommendation system never needs to actually look at the content directly, just at whether users like it or not.

There are problems with this approach, though, and a big one is the "cold start" problem–what if you have content in your system that few (or no) users have interacted with? Even though you might have information about the content available to you (like a text description, or, in this case, a poster/cover image), a collaborative system has no way to read this information, and so it can't recommend the content to users until people have found it organically first.

So, a cooler option is to build a system that actually understands the content it recommends. In this case, there was an existing project called Illustration2Vec that used a deep neural net to predict tags from anime images. These authors have built a hybrid recommendation system that uses collaborative filtering when there is a lot of data available, and then uses tag similarity (getting the tags by running Illustration2Vec on the posters/cover images) when data is sparser. It can also "explain" the results by telling you which tags it used to give you the recommendation.

The authors are using data from (and presumably, incorporating the resulting system into) a French anime/manga site called Mangaki (https://mangaki.fr/about/en).