Acorn with Liana Patel and Abdel Rodriguez – Weaviate Podcast #99
This podcast dives into all things ACORN! To set the stage, Vector Search with Filtering is very slow at scale for highly selective filters, such as a filter that composes <5% of the index. There have been a few approaches to combat this such as IVF^2, that stores the oracle partition index per filter, Window Search Tree, that has a clever strategy for continuous-valued filters, or Filtered DiskANN, that modifies the pruning heuristic based on unique filter membership.
The caveat with these approaches is that they require knowing the filters you want to search with in advance, and don't generalize well to arbitrary combinations of filters, such as `city` = "Boston" AND `num_bedrooms` < 3 OR ... -- they also typically add a significant amount of memory to the Vector Database.
ACORN instead proposes a neighbor of neighbors pruning and search heuristic that increases the connectivity of the subgraph induced by the filter! This enables super fast search with minimal indexing slow downs and added memory!I think this is such an exciting advancement for the field of Approximate Nearest Neighbor Search, and I can't wait to see the continued use and development of Vector Search with Filtering! Major thanks to Liana and Abdel for joining the podcast!!
YouTube: https://youtu.be/PxJ7FpbopKY
Spotify: https://spotifyanchor-web.app.link/e/zPsBAx0zIKb