Bloom filters: the niche trick behind a 16× faster API – Blog – incident.io

incident.io

11 pointsbirdculture4 comments

Splunk uses bloom filters to quickly skip buckets that don't contain keywords.

Wait until they find out that Postgres has native bloom filters [0].

0: https://www.postgresql.org/docs/current/bloom.html

I'm on the team who built this, I don't understand how you think these would be used. We need to use a b-tree index to give us ordered results that we then filter by the bloom column, how would an index on the bloom filter column help if we need the ordering?

Context is we didn't think these would work, and did look quite closely. We may have missed something though.

Upon re-reading the article, I must apologize - you’re correct, the native bloom extension doesn’t help here. I misread the constraints.

With that, hopefully it doesn’t come across as insincere or groveling to say this is an extremely cool way to solve the problem!

Reply on news.ycombinator.com