Qgrep Internals
zeux.io
zeux.io
But I agree. A prerequisite to any tool like this really need to be very good at keeping the index up to date with minimal user fuss and needs to be reliable. That's a hard engineering task.
Besides this, I did learn a trick or two from the blog.
Do other search tools like ag or ripgrep also maintain a cache? A lot of the optimizations he described could be done on none-cached data and I am curious to see how important the I/O and cache size bottleneck are.
Then there's the obvious left-out of time it takes for qgrep to populate its cache.
Seems like an apples and oranges comparison to me.
> Seems like an apples and oranges comparison to me.
This kind of misses the point. The OP is perfectly up front about this. It's not an apples to apples comparison in the sense that the tools are doing roughly similar algorithmic work internally, but it is a decent apples to apples comparison in terms of the user experience. That is, searches are much faster when you have a pre-computed index.
With that said, getting the user experience around keeping the index up to date is definitely a key component of this, and the article doesn't really explore that aspect. But I don't think index creation/update time should be included search benchmarks, since the whole point of a tool like this is that creating or updating the index should be a relatively cheap and infrequent thing compared to the searches executed against the index.
Thanks for sharing this link!
Testing your code on low cpu clock and disks with spinning parts. I'm pretty sure if we had this hardware in the 60s, computer science would have halted because people would have thought "oh why bother reducing this algorithm to nlogn where n^2 is computable in a perfectly reasonable time frame for the use cases we have?"
Nowadays, I can get by just fine with the search functionality in VSCode. (And I don't write VB.NET anymore.)