Hacker News Reader

Awesome work! How did you use Bloom filters to achieve low latency lookup?

I periodically download the 8GB com.zone root name file from Verisign and compress it into a set Bloom filters that allow for very fast lookup while using less memory than a textual hash. For fast responses, the filters are served over a UNIX domain socket to the WebSocket server on the backend.

That's a great approach I've thought of before -- so how big does the bloomified version end up being?

About 1GB to get very low error rates.

Would you mind sharing a bit about how the process to gain access to the root zone file works? I've seen the application form, but I'm curious how the whole thing goes from application to access. Any insights you can share?

I've been kicking around a few ideas, but didn't want to go through an involved process if it would only end in heartbreak. Does it require a specific use case as part of the application?

The process wasn't bad when I did it, actually a year or two ago. It took me this long to get around to doing anything with the data. :)

If you'd like to collaborate, I'm totally game. Send me a note.

Reply on news.ycombinator.com