TensorFlow Graph Neural Networks
blog.tensorflow.org
blog.tensorflow.org
We think a lot about security/fraud and digital journeys, where NN + xgboost are popular in general, and graph is used seperately (or upstream) for looking at broader structure. GNNs help blend these models. For example, in analyzing malicious user accounts (ex: misinfo on twitter), we already get many time/nlp/etc scores for whatever events/entities we look at, and use the social network structure to ensure better propagation/blending, similar to why boosting and ensemble methods became popular to beginwith. Feel free to DM if interested, we are quite excited by this space and working on some things here.
[1] https://www.cs.mcgill.ca/~wlh/grl_book/files/GRL_Book.pdf
There are important cases like maps and chemistry that take more specialized techniques, but we focus on events/logs/etc. So less to say on the niche stuff, even if those niches cover big use cases like "how google maps works" or "how google auto-designs their TPUs"
For the logs/events/transactions/clicks/devices/users/accounts cases, happy to chat, but maybe not as useful elsewhere :)
1. {x0, x1, ...} - nodes in the graph, say users
2. Bunch of edges like {x_i, x_j}, say social connections
3. Some raw or processed features on nodes, say the text of posts, age of the account or some nlp-based scores for posts
4. Some raw or processed features on the edges (in particular maybe some coloring)
Before: people would train various classifiers/regressors directly on the nodes and/or edges, then maybe use the graph structure to propagate the scores.
After: But instead you could train whatever objective you have from raw features on the edges and nodes, with some extra message passing between nodes and edges. For example train (some of) that nlp-based classifier together with the graph part. And the benefit would be that, for example, you can extract some signals from the NLP part that would be more useful in determining the properties of neighbors, but not necessarily as useful in determining the properties of the current node/edge.
Question - what's the maximum range of such message passing? Sounds a bit like an RNN, where the unroll depth can be an issue. Though in practice most graphs have a low average path length, so maybe this is not a particularly big problem.
Although if you start unrolling graphs you'll very quickly load ~everything, so I guess the training must be completely reworked (flush data to distributed storage frequently then shuffle for the next step) or you cannot unroll further than maybe a few steps.
Before: People might precompute graph scores ("pagerank", ...) and use as features for tabular NNs. Or use simpler and slow GNNs like GraphSAGE bc the domain fit was great (ex: Pinterest social recs)
After: heterogeneity and scale for graphs that fit in CPU RAM (1TB) w decent GPUs
Re:unrolling, yeah a bunch of papers there :) sampling, artificial jump edges, and adversarial techniques have been helping with aspects of generalization (far data, unbalanced data, ...)
> In the example below, we build a model using the TF-GNN Keras API to recommend movies to a user based on what they watched and genres that they liked.
> The code above works great, but sometimes we may want to use a more powerful custom model architecture for our GNNs. For example, in our previous use case, we might want to specify that certain movies or genres hold more weight when we give our recommendation.
A GNN can take into account everything you know about movies, including incomplete data. Therefore a GNN will see that user A likes everything with actor X, user B really wants the genre to be Y, user C likes actor Z, but only before 2000, and combinations of that. Therefore the GNN can do better, hell, it can even predict what movie properties would do well, which would be tough to get out of the embedding network.
You could encode all this data in your embedding, but this will be a much smaller and much more flexible network.
For example, in drug discovery you can treat any molecule and its biochemical properties as a graph so you can store the entire structure as a graph. So you make a million graphs of the biocompounds you know, a million graphs of random/similar compounds, and see which of the new million molecules the GNN will predict to work on disease X.
Long review here that might be closed access: https://academic.oup.com/bib/article/22/6/bbab159/6278145