Hacker News Reader

Elastic/OpenSearch have classically used BM25, but the latter recently added semantic search capabilities using neural embeddings in 2.4. Not sure about the former.

[1] https://opensearch.org/blog/opensearch-2-4-is-available-toda...

View on HN

ES and OS are desperately slow because based on the lucene vector search index. A dedicated vector database like Qdrant will be always a better choice https://github.com/qdrant/qdrant

Do we have any idea why lucene vector search underperforms? As of lucene 9.1 (and elastic 8.4), it runs the same sort of filtered/categorical HNSW that qdrant runs (https://lucene.apache.org/core/9_1_0/core/org/apache/lucene/...). Qdrant's benchmarking code (https://github.com/qdrant/vector-db-benchmark/blob/9263ba/en...) does use the new filtered ann query with elastic 8.4, so it appears to be a fair benchmark. Why is lucene/elastic so much slower? Is it a rust vs. java thing? Or some memory management issues?

Reply on news.ycombinator.com