Wonder if they had indexes on their SQLite tables?
Not seeing a mention of that in the article.
Not seeing a mention of that in the article.
In the article, we refer to this as "decimal lexical" formatting, where we're taking the IPs and making them integers but actually treating them as strings. We're doing this in both Redis with sorted sets and then in a single table in SQLite.
I was going to explain all this in the article, but it was too long already, so it will be a future blog post.
But about the formatting of the data, is it completely inherent to the rest of the system / unchangeable? Spontaneously I would have guessed that for example a bitfield in redis would have performed better. Did you test any other formattings?