Semantic Query Engines with Matthew Russo (MIT)
AI presents new Semantic Operators for our query languages. For example, we are all familiar with the WHERE filter. Now we have AI_WHERE, in which an LLM, or another AI model, computes the filter value without needing it to already be available in the database!
```sql SELECT * FROM podcasts AI_WHERE “Text-to-SQL” in topics ```
Semantic Filters are just the tip of iceberg, the roster of Semantic Operators further includes Semantic Joins, Map, Rank, Classify, Groupby, and Aggregation!
And it doesn’t stop there! One of the core ideas for Relational Algebra and how its influenced Database Systems is query planning and finding the optimal order to apply filters. For example, let’s say you have two filters, the car is red and the car is a BMW. Now let’s say the dataset only contains 100 BMWs, but 50,000 red cars!! Applying the BMW filter first will limit the size of the set for the next filter!
This foundational idea has all sorts of extensions now that LLMs are involved! This opportunity is giving rise to new query engines and declarative optimizers such as Palimpzest, LOTUS, and others!
I am SUPER EXCITED to publish the 131st episode of the Weaviate Podcast with Matthew Russo, a Ph.D. student at MIT!
So many interesting nuggets in this podcast, loved discussing these things with Matthew, and I hope you find it interesting!
YouTube: https://youtu.be/koPBr9W4qU0
Spotify: https://spotifycreators-web.app.link/e/ddUhVMmLoYb
Medium: https://medium.com/@connorshorten300/semantic-query-engines-with-matthew-russo-weaviate-podcast-131-131a42bbc521