Is anyone using this? It's from 2019.
The other one that I've always been curious of is Poincarré Embeddings [1] - where the embedding also has a hierarchical representation of the space.
There's issues with these becoming popular:
1. Querying the embeddings requires more math knowledge than just "lol cosine similarity". This also requires to write code for the query
2. You can often easily match the performance with regular embeddings by just adding dimensions and training more. So the advantage of exotic embeddings has to be with the information in the more complex mathematical abstraction.
So they need a killer usecase to become popular, it's hard to move the needle.
Do you think it makes sense to have a group of models, each with more ad-hoc embeddings, and coordinate them to respond according to the domain of the input?
Do multi-modal models use the same embedding type/structure for an image, sound, text?
The answer to the second is yes - it's all vector embeddings, and they're aligned to each other by finding a dataset that matches pairs (eg. images with captions)
The real use for exotic embeddings will have to be in analyzing the embeddings themselves I think, otherwise it's easier to shove normal vectors downstream into other models.
In this context "knowledge graph" means a representation of knowledge independent of it's serialization format (ie, RDF vs whatever).
It's usable wherever you have a key value (key:value of relationship) or a triple (key:relationship:something).
This is probably 85% of everything that is stored in a database, with the exception being paths that vary enough that you can't use the path as a key or relationship. In particular, most trees are suitable if you use collapse the path down to a single relationship.
I've done a bunch of work on graph embedding. They are very effective for use in anything that can be thought of as a recommendation system ("this person would like these books") or similarity ("this person is similar to these people").
Back when I was working on them I found Starspace wonderfully easy and effective: https://github.com/facebookresearch/StarSpace