Is there any info on query performance? And to builders, did you get better results by incorporating semantic search?
This engine used a neural network to train an autoencoder that crunches down the word counts for thousands of words to a moderate dimensional vector, say n=50. This captures correlations between words such that similar documents are more consistently close in the embedding space than they are in the very high dimensional word vector space.
This kind of system does not improve short queries (<10 words) but is great for "more like this" queries centered on a document and taking paragraph you wrote describing an invention and finding prior art.
We used the TREC evaluation methodology, public data, our proprietary data, and the opinions of users to conclude our product was much better than a simple baseline search engine and our competitors.
But I should clarify, did we get enough of a comprehension boost from transformer based Pretrained LLMs to successfully interpret query intent and find related items?
I've seen plenty of blog posts where somebody did something poorly motivated with an embedding and had a search engine that worked but didn't do any real evaluation so you don't know if it is better or worse than a simple search engine that uses, say, Okapi BM25.
For example i expect that searching "a snarky blog post" or "uplifting people"... wouldn't work that great