4,973 karma · joined October 20, 2014
I love chess, deadlifting, and Elden Ring.
Twitter: https://x.com/frankzliu
LinkedIn: https://www.linkedin.com/in/fzliu
I'll have to dig out the notebook that I created for this, but I'll try to post it here once I find it.
Just took a look at your docs and product page as well. Keep up the great work!
That's why we built attribute filtering into Milvus via a Mongo-esque interface. No SQL and not as performant as an RDBMS, but it's an option: https://milvus.io/docs/hybridsearch.md
The most commonly used one I've seen is HNSW, which greatly speeds up search at the expensive of some extra memory consumption. Recall is also strong - for many large-scale use cases you can get 95%+ with properly tuned parameters.
Here's a highly simplified example: if my query was "fields related to computer science", a semantic engine could return "statistics" and "electrical engineering" while avoiding things such as "social science" and "political science".
For those interested, here's a comparison with other open source vector databases: https://zilliz.com/comparison. For those who don't want to be burdened with installing and maintaining a local database, there's a managed service available as well: https://zilliz.com/cloud.
1) Embedding crash course https://developers.google.com/machine-learning/crash-course/...
2) What is a vector database? https://zilliz.com/learn/what-is-vector-database
3) Introduction to vector similarity search https://zilliz.com/blog/vector-similarity-search
4) ANN benchmarks http://ann-benchmarks.com
5) Unstructured data ETL https://github.com/towhee-io/towhee
Once you reach a certain scale, it's helpful to potentially use distributed querying and/or different index types, even if you have a fairly static dataset. You can check out a billion-scale search benchmark we recently did here: https://zilliz.com/resources/milvus-performance-benchmark (you'll need to supply your email unfortunately). Here's the framework we used as well: https://github.com/zilliztech/vectordb-benchmark
Milvus (https://milvus.io) and Feast (https://feast.dev/) are two of the most well known vector databases and feature stores, respectively.
Curious about both performance/QPS and scale/# of vectors.
I've had the same experience with cheap (Chinese) microcontrollers - power consumption is usually higher across all modes.
Google/DeepMind might have an RL-based chatbot equal to or even better than ChatGPT, but OpenAI has achieved the most due to its ability to ship.
[I'm a part of the Milvus community.]
With that being said, one of the main challenges ahead will be multimodal learning. We're sort-of there combining text with visual data, but there are many other modalities out there as well.
This is only partially true. When digital chips boot up, gate outputs are in an indeterminate state. The reset sets them to know initial (and valid) values/bits.
Good stuff!