This is pretty cool, but what are the scaling limits of a pgvector-based embeddings storage solution? 1 TB of embeddings? 100 TB? Is pgvector suitable for large scale installations?
Blog author. I've done some separate testing on storing ~500GB of embeddings (~1B embeddings) in a partitioned table. The partition key was built using IVFFLAT as a "coarse quantizer" (in this case, sampling the entire dataset and finding K means), storing the mean vectors in a separate table, and then loading each vector into the partition with closest center. After that, I built an IVFFLAT index on each partition. With the indexes, this added up to ~1TB storage. This was primarily a "is it possible test" vs. thorough benchmarking.
The scalability characteristics for HNSW haven’t been properly benchmarked yet. If it’s anything like ivfflat then it should be reasonably predictable based on memory size: https://supabase.com/blog/pgvector-performance
I think 100TB is getting more into “sharded postgres” territory