emilfroberg

Great article Lukas! A key takeaway here is that Llama was the best LLM. I wouldn't have expected that.

(Disclaimer, Lukas and I are co-founders)

Thanks for your input, I've only tried Chroma a little bit so far and had a pretty good experience. What they also have going for them is a big community on discord that can be helpful.

Yeah, maybe they will.. But for now, the best options are the purpose-built vector databases, so why not use them?

Many of them are open source and you can host them yourself. That would make it more cost effective. Also someone mentioned https://turbopuffer.com/. That seems like a good alternative if you're looking for something economical.

I quickly took a look at the redisearch ANN Benchmarks and they seem to stack up against the others (more or less same level as Milvus) in the comparison when it comes to QPS and Latency.

Yeah, that's the difference we've seen according to the QPS for the ANN Benchmarks. The same story seems to be true for other datasets too. We're looking at a 0.9 recall.

The way you explain hybrid search aligns with my understanding. Pinecone has a good article about it here https://www.pinecone.io/learn/hybrid-search-intro/. From my understanding, all vector DBs support this.

Happy to connect. The benchmark numbers are mostly from ANN Benchmarks. For my use case, the nytimes-256 dataset was most relevant so I used that for the QPS benchmark. I also took a look at the benchmarks you've made at https://qdrant.tech/benchmarks/ and there qdrant seems to be outperforming many others. If I've gotten something wrong here, I'm glad to update the article :)

Someone else also pointed out that Vespa was missing. I'll have to look in to it and add it to the article!

Me too! Couldn't find a lot of information on it yet, but I might have to try it myself to get some benchmarks

Turbopuffer looks like something I would consider. And the pricing looks to be lowest on the list from what I can see

Txtai looks interesting, maybe you could help me collect some of the comparision parameters for it?

Vespa looks interesting, hadn't seen it before but will definitely take a look at it

I made this table to compare vector databases in order to help me choose the best one for a new project. I spent quite a few hours on it, so I wanted to share it here too in hopes it might help others as well. My main criteria when choosing vector DB were the speed, scalability, dx, community and price. You'll find all of the comparison parameters in the article.