HNHacker News
TopNewBestAskShowJobs

fzliu

4,973 karma · joined October 20, 2014

Applied ML @ Voyage AI (https://voyageai.com).

I love chess, deadlifting, and Elden Ring.

Twitter: https://x.com/frankzliu

LinkedIn: https://www.linkedin.com/in/fzliu

submissionscomments
fzliu··on Vespa.ai is spinning out of Yahoo as a separate company
We have a "Vector Database 101" series that covers vector search and vector indexes as well: https://zilliz.com/learn/what-is-vector-database

I've been meaning to dive into SCaNN and DiskANN as well, but haven't gotten around to it yet.

fzliu··on Vespa.ai is spinning out of Yahoo as a separate company
I was at Yahoo almost a decade ago, when vector search within Vespa was first being rolled out in production use cases. It was already serving similarity search requests for Flickr back then.

Even though I'm with Zilliz/Milvus now, I wholeheartedly support and recommend folks check out and try Vespa. Congrats to the Vespa team!

EDIT: For folks on Twitter, you should follow Jo (https://twitter.com/jobergum) from Vespa if you aren't already. Great combo of technical content, hot takes, and vector database memes!

fzliu··on Choosing vector database: a side-by-side comparison
Shameless self-plug for milvus-lite:

   $ pip install milvus
   $ python
   >>> import milvus
   >>> milvus.start()
fzliu··on Vector databases: analyzing the trade-offs
Shameless self-plug for our embedded vector database milvus-lite (https://github.com/milvus-io/milvus-lite):

    pip install milvus
fzliu··on Consensus: Use AI to find insights in research papers
I'll play DA here - there's quite a bit of engineering surrounding these apps that can appear hidden to folks from the outside looking in. Various levels of prompt engineering and in-context learning might be necessary to get optimal results, and this could mean significantly more complexity at the application level.
fzliu··on On-disk HNSW index for Postgres with pg_embedding
Thanks for sharing. How does this compare with DiskANN (https://zilliz.com/blog/diskann-a-disk-based-anns-solution-w...) or HNSW-IF (https://blog.vespa.ai/vespa-hybrid-billion-scale-vector-sear...)?
fzliu··on Ask HN: In which areas have you compared 3+ tools and formed strong preferences?
What kind of queries over structured data do you use Postgres for? We're looking to see how we should broaden our metadata filtering to potentially support usage patterns for developers like yourself.
fzliu··on LLMs Unleashed: The Power of Fine-Tuning
This paradigm applies for much more than just answering questions - I've found the in-context learning capabilities to be very relevant as well. For example, you can query a vector database and input that information into an LLM and ask it to make a prediction across the input data.

https://zilliz.com/blog/ChatGPT-VectorDB-Prompt-as-code

fzliu··on Introduction to vector similarity search (2022)
I would not use word2vec in any application today - I used it in the blog post because it's a well-known model and because the embeddings it generates are static, i.e. one token always maps to the same embedding regardless of context.

If you want to create embeddings at the sentence level, a good place to start is SBERT: https://www.sbert.net/

fzliu··on Chiplet ASIC supercomputers for LLMs like GPT-4
Just skimmed the paper. Seems to me like this paper wants to optimize transformer inference e2e, i.e. from ASIC level all the way to cloud.

I'm not exactly convinced though, since all the results seem to be purely theoretical or simulated. I would've liked to see a prototype built across several FPGAs with clock speeds extrapolated for ASICs.

fzliu··on Introduction to vector similarity search (2022)
You might enjoy milvus-lite: https://zilliz.com/blog/exploring-magic-vector-databases-jup...

   pip install milvus
fzliu··on Introduction to vector similarity search (2022)
I would not use Elastic for vector search due to its architectural limitations and poor performance when conducting vector search. https://zilliz.com/benchmark
fzliu··on Introduction to vector similarity search (2022)
If you're just starting out, I'd use sentence-transformers for calculating embeddings. You'll want a bi-encoder model since they produce embeddings. As the author of the blog, I'm partial towards Milvus (https://github.com/milvus-io/milvus) due to its enterprise and scale, but FAISS is a great option too if you're just looking for something more local and contained.

Milvus will perform vector similarity search for you - all you need to do is give it a query vector.

fzliu··on Introduction to vector similarity search (2022)
word2vec is seminal and outputs static embeddings - fairly easy for beginners to understand. The embeddings for each token change relative to context in attention-based MLMs and I figured it might be confusing for an introductory blog.
fzliu··on AI for AWS Documentation
We built something very similar, except for other open-source projects rather than AWS documentation: https://osschat.io
fzliu··on AI for AWS Documentation
Hallucination is unfortunately inevitable when it comes to any autoregressive model, even with RAG. You can minimize hallucination by prompting, but you'll still see some factually incorrect responses here and there (https://zilliz.com/blog/ChatGPT-VectorDB-Prompt-as-code).

I unfortunately don't think we'll be able to solve hallucination anytime soon. Maybe with the successor to the transformer architecture?

fzliu··on Ask HN: Any hardware startups here?
We did (and still do) indoor localization (essentially GPS for indoor spaces). We started off building our own hardware and during the prototyping phase, we built a quad-channel software defined radio: https://github.com/fzliu/osdr-q10

My 2¢: don't do a hardware startup. Iteration cycles are long and expensive, funding is minimal, and if you're successful you'll be squeezed out of the market by giants such as Intel.

fzliu··on Emerging architectures for LLM applications
Hell yeah! Feel free to reach out if you need any help.
fzliu··on Emerging architectures for LLM applications
Bit of self-promotion, but Milvus (https://milvus.io) is another open-source vector database option as well (I have a pretty good idea as to why it isn't listed in a16z's blog post). We also have milvus-lite, a pip-installable package that uses the same API, for folks who don't want to stand up a local service.

    pip install milvus
Other than that, it's great to see the shout-out for Vespa.
fzliu··on The Secret Sauce behind 100K context window in LLMs: all tricks in one place
I don't think chunking is the most optimal approach either. I can envision a future where embeddings have variable length and comparing two variable-length embeddings would require a more complex similarity metric. Vector databases will need to adjust to this reality.
fzliu··on Vector Database in a Jupyter Notebook
Although it's common to use vector databases in conjunction with LLMs, I like to talk about vector databases in the context of unstructured data, i.e. any data that you can vectorize with (or without) an ML model. Yes, this includes text, but it also includes things like visual data, molecular structures, and geospatial data.

For folks who want to learn a bit more, there are examples of vector database use cases beyond semantic text search in our bootcamp: https://github.com/milvus-io/bootcamp

fzliu··on Vector Database in a Jupyter Notebook
Hola Jeff. Let's host a meetup together.
fzliu··on VanillaNet: The power of minimalism in deep learning
Transformers don't seem to optimal as an architecture for vision tasks, but I don't think pure convnets are either. Either way, having the right data is 10x as important.
fzliu··on Show HN: YouTube Full Text Search – Search all of a channel from the commandline
Here's one that may be helpful for you: https://zilliz.com/blog/using-vector-database-search-white-h...
fzliu··on Product quantization for vector search
AFAIK pgvector only supports IVF, no PQ (nor HNSW).
fzliu··on Product quantization for vector search
A naive PQ index that trains the quantizer once and reuses the same codebook for recent data definitely has this problem. We (Milvus/Zilliz) break up all the vectors into multiple indexes at certain checkpoints which we call segments. This doesn't fully solve the problem of degraded performance with PQ or hybrid PQ indexes, but certainly helps mitigate it.

More info here: https://milvus.io/docs/glossary.md#Segment

fzliu··on Product quantization for vector search
Just read the abstract and intro for the link you posted - thanks for sharing.

As great as HNSW and other graph-based indexes are, I think PQ and other encoder/decoder-based methods are still incredibly important for ANN search in general. In particular, it should be possible to learn some sort of joint encoding with neural networks targeted towards different modalities.

fzliu··on Product quantization for vector search
For folks interested in the rest of our vector search 101 series, here's a link: https://zilliz.com/blog?tag=39&page=1
fzliu··on Numbers every LLM developer should know
AFAIK for over-parameterized models, performing quantization or any other form of compression won't reduce accuracy by much (don't quote me on this though).
fzliu··on 100K Context Windows
I'm also not entirely convinced by "huge" context models just yet, especially as it relates to fuzzy knowledge such as overarching themes or writing style.

In particular, there are 0 mentions of the phrase "machine learning" in The Great Gatsby, so adding one sentence that introduces the phrase should be easy for self-attention to pick out.

← PreviousPage 2 of 6Next →