HNHacker News
TopNewBestAskShowJobs

fzliu

4,973 karma · joined October 20, 2014

Applied ML @ Voyage AI (https://voyageai.com).

I love chess, deadlifting, and Elden Ring.

Twitter: https://x.com/frankzliu

LinkedIn: https://www.linkedin.com/in/fzliu

submissionscomments
fzliu··on PaLM 2 Technical Report [pdf]
I don't understand how this can be considered a technical report. No information on model architecture, distributed training methodology, or optimizations. The "Training dataset" section is a pathetic 0.5 pages long.

Come on, Google.

fzliu··on Why I'm done with Mouser Electronics
Not sure how you arrived at this conclusion. Mouser and Digi-Key are nearly identical in terms of UI. Search functionality on top followed by products & manufacturers and a list of products on the left.

I'm with you on that second point though - both websites look like they were designed by hardware engineers rather than "omg-UI-UX-is-so-important" SWEs.

fzliu··on Ask HN: What is new in algorithms and data structures these days?
Most commonly used ANN algorithms today are clever optimizations atop "old" data structures. DiskANN is a recent favorite: https://milvus.io/blog/2021-09-24-diskann.md
fzliu··on Abusing vector search for texts, maps, and chess
Totally agree. It takes _a lot_ to go from Hnswlib to a full-fledged vector database.

Here's an architecture diagram for a production-ready vector database https://milvus.io/docs/architecture_overview.md. Not exactly something you can build in a month.

fzliu··on Giving GPT “Infinite” Knowledge
Encoder-decoder (attention) architectures still have a tough time with long-range dependencies, so even with longer context lengths, you'll still need a retrieval solution.

I agree that there's probably a better solution than pure embedding-based or mixed embedding/keyword search, but the "better" solution will still be based around semantics... aka embeddings.

fzliu··on Giving GPT “Infinite” Knowledge
Not sure what you mean by dynamic dictionary, but the embedding tree you mention is already freely available Milvus via the Annoy index.
fzliu··on Billion-Scale Approximate Nearest Neighbor Search [pdf]
SCaNN is coming to Milvus very soon (we've seen some demand from the community). Stay tuned.
fzliu··on eBay’s Fast Billion-Scale Vector Similarity Engine
We've seen Milvus used in a variety of recommender systems running in production.
fzliu··on eBay’s Fast Billion-Scale Vector Similarity Engine
Agreed. There's a lot of great index types out there - HNSW is incredible, but algorithms such as ScANN (and PQ) have their place in the ecosystem.

Tree-based vector indexes aren't bad either, especially if we can find a way to make the random projections more efficient.

fzliu··on Fixing Hallucination with Knowledge Bases
Human features such as faces and fingers are easy for us humans to parse because our attention is instantly drawn to them, and we are highly sensitive to "noise" for that particular domain. AI-generated content of a three-toed animal, for example, that is pictured with four toes might slip a lot of people's minds.

I unfortunately don't see how vector database would be able to help with this. The knowledge bases that they provide are meant to enable large-scale retrieval and vector search. One way to fix the problem you mention is perhaps to increase the proportion of features that draw human attention (e.g. faces) in the training dataset.

fzliu··on Fixing Hallucination with Knowledge Bases
Milvus (https://milvus.io) and Vespa (https://vespa.ai) are great choices if you're looking for hardened, scalable, and production-ready vector databases.

We (Milvus) also have `milvus-lite` if you'd like something pip installable: python3 -m pip install milvus

fzliu··on Re-implementing LangChain in 100 lines of code
LangChain is meant to reduce/remove the amount of boilerplate code needed to build a lot of applications with LLMs. I see your point, but I still think LangChain is useful for a particular segment of early stage developers.
fzliu··on Google “We have no moat, and neither does OpenAI”
Vector databases such as Milvus are only there to help reduce/minimize hallucinations rather than get rid of them completely. Until we have a model architecture that can perform completion from the _prompt only_ rather than pre-training data, hallucinations will always be present.
fzliu··on Avoiding hallucinations in LLM-powered applications
The idea of injecting domain knowledge into LLMs can certainly help, but they don't fix the problem entirely. There are still plenty of opportunities to hallucinate - for example, ChatGPT still regards the phrase "LLM" to refer to a law degree, and domain knowledge won't fix that unless it is explicitly spelled out in the prompt. This article is provides a similar overview: https://zilliz.com/blog/ChatGPT-VectorDB-Prompt-as-code (I work at Zilliz).
fzliu··on Ask HN: Who is hiring? (May 2023)
Zilliz | Onsite/Hybrid (Redwood City, CA, USA) - https://zilliz.com/

Zilliz is the company behind Milvus, the world's most popular open-source vector database. We're hiring engineers, developer advocates, product marketing, and product managers. You can view and apply for open roles at https://zilliz.com/careers, or feel free to reach out to me directly.

fzliu··on Creating AI assistant with GPT and Ruby and Redis using embeddings
Even with an incredibly long context window (say, 1M tokens), attention still suffers from a problem with long-term dependencies. This is probably why OpenAI hasn't publicly released their 32k token length model just yet.
fzliu··on The Inner Workings of Distributed Databases
Here's a page on the architecture of a distributed vector database (Milvus) for anybody interested: https://milvus.io/docs/architecture_overview.md
fzliu··on Reduce your GPT bill via semantic caching
Certainly. Every cache (including a semantic cache like GPTCache) needs an eviction policy to prevent the size of the cache from exploding. A single line is the smallest unit a cache can hold - for a semantic cache, that means a single pair of unstructured data such as text/text or text/image.
fzliu··on Sam Altman: OpenAI is not training GPT-5 and "won't for some time"
OpenAI may not be training GPT-5, but Sam didn't say anything about GPT-4.5.
fzliu··on Show HN: GPTCache – Redis for LLMs
The current implementations use Milvus and FAISS, but we have opens for other open-source vector databases as well.
fzliu··on GPTCache: Slash Your LLM API Costs by 10x
Keep up the awesome work. I've run across this problem myself - I somehow used $20 just testing a small demo I made with GPT-3.5.

As most ML is inherently probabilistic, it seems reasonable to make an LLM cache both semantic and _stochastic_, i.e. you wouldn't want the same answer every time you use "pick me a color" as prompt. Injecting the original LLM (GPT, Bard, etc) response as prompt for alpaca or some other model could make this cache virtually invisible.

fzliu··on Segment Anything Model (SAM) can "cut out" any object in an image
Computer vision seems to be gravitating heavily towards self-attention. While the results here are impressive, I'm not quite convinced that vision encoders are the right way forward. I just can't wrap my head around how discretizing images, which are continuous in two dimensions, into patches is the most optimal way to do visual recognition.

What's preventing us from taking something like convnext or a hybrid conv/attention model and hooking that up to a decoder stack? I fee like the results would be similar if not better.

EDIT: Clarifying that encoder/decoder refers to the transformer stack, not an autoencoder.

fzliu··on Launch HN: Type (YC W23) – AI-powered document editor
I think the core idea is strong, but it's missing several features that could differentiate it as a product:

1) adding an "advanced" tab which enables the user to input more information about the document, e.g. formatting, length, etc...

2) learning the user's writing patterns and style - I'm not sure if this can be done by prompting, but my guess is that there are methods that you can use to steer the prose towards the user's default style

3) plagiarism detection - this would require a large document store and could be implemented using fuzzy/semantic search via Milvus followed by string matching

Keep up the good work.

fzliu··on LangChain Announces 10M Seed Round
This is the path forward that I see for Langchain commercialization as well. As we begin to see more and more products built atop LLMs, these hosted services could form the backbone for a lot of those applications.

Disclaimer: We - Milvus (https://milvus.io) and Zilliz (https://zilliz.com) - and we have integrations as a vector store in Langchain.

fzliu··on Ask HN: Are you still doing intermittent fasting?
I did 18/6 for quite a while before switching back to 16/8 due to a busy work schedule. IF is a long-term play - having the occasional burger or some beers late at night on Friday or Saturday won't affect your results much. Eat healthy, and keep at it.
fzliu··on ChatGPT – The Revolutionary Bullshit Parrot
Even though GPT-4 "fixes" many of the problems pointed out in this blog post, I question whether or not some of the underlying issues associated with transformers (and the attention mechanism in particular) can be overcome with with more sparse attention and deeper/wider models.

e.g. GPT-3.5 might not be able to write simple poems correctly, but does GPT-4 solve the disease or the symptom (by masking it with a bigger model and more data)? I'm not entirely convinced.

fzliu··on Could you train a ChatGPT-beating model for $85k and run it in a browser?
I was a bit skeptical about loading a _4GB_ model at first. Then I double-checked: Firefox is using about 5GB of memory for me. My current open tabs are mail, calendar, a couple Google Docs, two Arxiv papers, two blog posts, two Youtube videos, milvus.io documentation, and chat.openai.com.

A lot of applications and developers these days take memory management for granted, so embedding a 4GB model to significantly enhance coding and writing capabilities doesn't seem too far-fetched.

fzliu··on Faiss: A library for efficient similarity search
Vector indexes only form a component within vector databases - you'd want the database bits and bobs (scalability, replication, caching, etc) surrounding it in addition to the index itself. Milvus, for example, supports flat indexes - as far as I know, Weaviate could support flat indexing as well if it doesn't do so already.

Most vector databases focus on ANN because of scale. Once you get to around a million vectors or so, it becomes prohibitively expensive to perform brute-force querying and search.

fzliu··on Faiss: A library for efficient similarity search
This is typically done by taking the activations from a neural network. Activations from layer depths are generally preferred since they are more representative.
fzliu··on Faiss: A library for efficient similarity search
The design of Milvus 2.x follows this paper we published a while back: https://arxiv.org/abs/2206.13843. In short, we used Pulsar to implement the write-ahead log, which provides coordination and a single source of truth across all Milvus components.

You're right in that it's a bit heavyweight, so we're working to see how we can make pub/sub and other cluster components lighter and more efficient overall.

← PreviousPage 3 of 6Next →