HNHacker News
TopNewBestAskShowJobs

fzliu

4,973 karma · joined October 20, 2014

Applied ML @ Voyage AI (https://voyageai.com).

I love chess, deadlifting, and Elden Ring.

Twitter: https://x.com/frankzliu

LinkedIn: https://www.linkedin.com/in/fzliu

submissionscomments
fzliu··on You probably shouldn't use OpenAI's embeddings
I'm curious about this as well. There are potentially many different versions of embedding models used in production and correlating different versions together could be very important.
fzliu··on You probably shouldn't use OpenAI's embeddings
I've done some quick-and-dirty testing with OpenAI's embedding API + Zilliz Cloud. The 1st gen embeddings leave something to be desired (https://medium.com/@nils_reimers/openai-gpt-3-text-embedding...), but the 2nd gen embeddings are actually fairly performant relative to many open source models with MLM loss.

I'll have to dig out the notebook that I created for this, but I'll try to post it here once I find it.

fzliu··on Faiss: A library for efficient similarity search
Faiss is a wonderful vector search library - in particular, the ability to do hybrid indexes e.g. IVF-PQ, IVF-SQ is great. We (https://milvus.io) use it as one of the indexing options (along with Annoy, Nmslib, and DiskANN) to power our vector database.
fzliu··on Command line functions around OpenAI
Beyond artificially increasing prompt length, vector DBs also allow you to inject domain knowledge into autoregressive LMs.
fzliu··on Launch HN: Metal (YC W23) – Embeddings as a Service
Great to hear that you're considering Milvus. Feel free to reach out if you ever have any questions/comments/concerns.

Just took a look at your docs and product page as well. Keep up the great work!

fzliu··on Vector database built for scalable similarity search
Redis does fairly well single-threaded - iirc it does better than Milvus. The problem is, once you move to larger applications and want higher performance, its architectural limitations make it impossible to scale.
fzliu··on Vector database built for scalable similarity search
Totally with you on that - it's annoying to need multiple pieces of infrastructure, especially for solo developers or small teams. Often times, you want to filter further based on scalar fields/metadata such as timestamps, strings, numeric values, etc...

That's why we built attribute filtering into Milvus via a Mongo-esque interface. No SQL and not as performant as an RDBMS, but it's an option: https://milvus.io/docs/hybridsearch.md

fzliu··on Vector database built for scalable similarity search
One key differentiator of Milvus is the ability to use a variety of different indexing algorithms. Some will consume very little memory and provide significant speedup at the expense of recall, while others are more powerful at the expensive of compute time or memory consumption.

The most commonly used one I've seen is HNSW, which greatly speeds up search at the expensive of some extra memory consumption. Recall is also strong - for many large-scale use cases you can get 95%+ with properly tuned parameters.

fzliu··on Vector database built for scalable similarity search
Traditional search such as ES and Lucene rely primarily on bag-of-words retrieval and keyword matching e.g. BM25, TF-IDF, etc. Vector databases such as Milvus allow for _semantic_ search.

Here's a highly simplified example: if my query was "fields related to computer science", a semantic engine could return "statistics" and "electrical engineering" while avoiding things such as "social science" and "political science".

fzliu··on Vector database built for scalable similarity search
Yup! Zilliz Cloud (managed Milvus) provides free credits for you to get started with a full-service (no throttling or other limitations) PoC as well.
fzliu··on Vector database built for scalable similarity search
Milvus is completely open source (https://github.com/milvus-io/milvus) and supports a variety of index types (https://milvus.io/docs/overview.md#Index-types) and support various consistency levels, scalar/metadata filtering, and time travel. We started working on Milvus back in 2018, with 2.0 being released in January 2022 (https://github.com/milvus-io/milvus/releases/tag/v2.0.0).

For those interested, here's a comparison with other open source vector databases: https://zilliz.com/comparison. For those who don't want to be burdened with installing and maintaining a local database, there's a managed service available as well: https://zilliz.com/cloud.

fzliu··on Vector database built for scalable similarity search
You'll have to manually compute the mean after retrieving the nearest neighbors. Automatic fast averaging could be an interesting use case for clustering or using a vector database to generate training though.
fzliu··on Vector database built for scalable similarity search
Not yet, but this functionality should be coming soon. We're currently working on adding the capability to call third party embedding APIs directly from a Zilliz Cloud instance.
fzliu··on Vector database built for scalable similarity search
I work at Zilliz (https://zilliz.com) and am a part of the Milvus community. Here are some other resources in case anybody's interested in learning more about embeddings, vector search, and vector databases:

1) Embedding crash course https://developers.google.com/machine-learning/crash-course/...

2) What is a vector database? https://zilliz.com/learn/what-is-vector-database

3) Introduction to vector similarity search https://zilliz.com/blog/vector-similarity-search

4) ANN benchmarks http://ann-benchmarks.com

5) Unstructured data ETL https://github.com/towhee-io/towhee

fzliu··on Open-Source GPT-4 Platform for Markdown
If you have a small number of fixed documents e.g. <100k or so, then I agree that pickling the vectors or storing them as bytearrays would work better.

Once you reach a certain scale, it's helpful to potentially use distributed querying and/or different index types, even if you have a fairly static dataset. You can check out a billion-scale search benchmark we recently did here: https://zilliz.com/resources/milvus-performance-benchmark (you'll need to supply your email unfortunately). Here's the framework we used as well: https://github.com/zilliztech/vectordb-benchmark

fzliu··on ChatGPT Plugins
+1, it's great to see OpenAI being active on the open source side of things (I'm from the Milvus community https://milvus.io). In particular, the vector stores allow the ability to inject domain knowledge as a prompt into these autoregressive models. Looking forward to seeing the different things that will be built using this framework.
fzliu··on GPT-4
One way to get around context length is to perform embedding and retrieval of your entire corpus. Langchain (https://langchain.readthedocs.io/en/latest/) and Milvus (https://milvus.io) is one of the stacks you can use.
fzliu··on Large language models are having their Stable Diffusion moment
Would like to bring up LangChain as well : https://langchain.readthedocs.io/en/latest/. We recently integrated Milvus (https://milvus.io) into LangChain, so you'll be able to store and process billions of documents.
fzliu··on Pgvector: Open-source vector similarity search for Postgres
Here's a demo using Milvus if you're interested: https://molsearch.milvus.io/.
fzliu··on A Critical Field Guide for Working with Machine Learning Datasets
Vector databases in general are good for storing large amounts of unstructured data by first converting them into embeddings via ML models. There's also feature stores, which store and organize features for later use in model training or predictive analytics. Feature stores generally come in _before_ models get trained, while vector databases generally come _after_ (i.e. they use trained models).

Milvus (https://milvus.io) and Feast (https://feast.dev/) are two of the most well known vector databases and feature stores, respectively.

fzliu··on DuckDuckGPT brings ChatGPT to search results
Good stuff! Just out of curiosity, what vector database are you using.
fzliu··on Storing OpenAI embeddings in Postgres with pgvector
First time I've heard of pgvector - for folks with experience, how does it compare to other ANN plugins (i.e. Redis https://redis.io/docs/stack/search/reference/vectors/) and purpose-built vector databases (i.e. Milvus https://milvus.io)?

Curious about both performance/QPS and scale/# of vectors.

fzliu··on The cheapest flash microcontroller you can buy is actually an Arm Cortex-M0+
> The PY32F030 parts advertise a typical low-power stop mode current consumption of 4.5 µA when using a 1.0V setting with the internal regulator, and I was able to get the part to drop down to 5 µA in my testing. Meanwhile, the STM32F030 uses less than half of that in stop mode

I've had the same experience with cheap (Chinese) microcontrollers - power consumption is usually higher across all modes.

fzliu··on Show HN: I trained an AI model on 120M+ songs from iTunes
Agreed 100%. Folks often forget that a great model is only a tiny part of the equation - strong execution and a solid demo are also critical.

Google/DeepMind might have an RL-based chatbot equal to or even better than ChatGPT, but OpenAI has achieved the most due to its ability to ship.

fzliu··on Same.energy: Image Search by Similarity
Related: https://milvus.io/milvus-demos/reverse-image-search/

[I'm a part of the Milvus community.]

fzliu··on Show HN: Turning books into chatbots with GPT-3
Might be usable for low column count tabular data, but it would be pretty terrible for any other semantically dense modality e.g. video, molecules, geospatial, etc.
fzliu··on Show HN: Turning books into chatbots with GPT-3
If you sprinkle in a bit of infrastructure, I think we're already there. The ability to distill a variety of content into vectors and perform approximate nearest neighbor search (shameless plug: https://milvus.io) across all of them can really help power a lot of these applications. With the retrieved vectors, you could match questions with answers or create a reverse index to the original content to perform summarization.

With that being said, one of the main challenges ahead will be multimodal learning. We're sort-of there combining text with visual data, but there are many other modalities out there as well.

fzliu··on Reasons Python Sucks (2018)
This feels like it was written by someone who has been writing C code their entire life.
fzliu··on What happens when a CPU starts
> This is because when the power supply is first powering up, even if it only takes a second or two, the CPU has already received "dirty" power, because the power supply was building up a steady stream of electricity. Digital logic chips like CPUs require precise voltages, and they get confused if they receive something outside their intended voltage range.

This is only partially true. When digital chips boot up, gate outputs are in an indeterminate state. The reset sets them to know initial (and valid) values/bits.

fzliu··on Show HN: I built Haystack – your own google for scattered workplace knowledge
As someone who uses a multitude of workplace apps myself, this is amazing. What kind of model are you using, and do you have plans to provide a service based off this?

Good stuff!

← PreviousPage 4 of 6Next →