If you want to create embeddings at the sentence level, a good place to start is SBERT: https://www.sbert.net/
If you want to create embeddings at the sentence level, a good place to start is SBERT: https://www.sbert.net/
Sbert needs to be finetuned to get anwhere near good results if the training objective deviates from the original one. Word2Vec can be estimated from collections of tokens in an unsupervised fashion, so it definitely has its place even today.
I guess what still doesn't add up in my mental model is how people seem to assume that the query embedding vector must somehow automatically be near good answer embedding vectors. I would have assumed that some conditions have to be met to make this true and I would be interested into advice in that regard, or alternatively an explanation why I'm wrong with my assumption.
Another thing that is not clear to me: Is there query directly fed into e.g. SBERT or should I ask an LLM to transform the query into something more suitable, like turning the question into a proposition?
Asked more abstractly: In a vector space like SBERT's, can I expect questions and answers about the same topic to lie near each other? Especially will the correct answers lie near their question?
This not how 99% of embedding models work(though you can train for specific tasks) but as it turns out such a thing is possible and is beneficial.
Instruct tuned embedding model here - https://instructor-embedding.github.io/