I'm the author of ag. Sift has been mentioned to me a few times, and I finally felt the need to comment on it.

Sift is a neat tool and I hope people get a lot of use out of it. That said, I don't think the chosen benchmarks are indicative of real-world performance. For example, I just downloaded the latest Linux kernel source, built it, and ran sift and ag on it. This is on a Core i7 4770K with 32GB of RAM and an Intel 330 series SSD. Times are medians of 5 runs each. FS cache was hot.

    ggreer@lithium:~/Downloads/linux-4.3% du -sh                                      
    12G .
    ggreer@lithium:~/Downloads/linux-4.3% time ~/code/sift/sift -n --group line_number
    ...
    ~/code/sift/sift -n --group line_number  10.03s user 1.59s system 625% cpu 1.856 total
    ggreer@lithium:~/Downloads/linux-4.3% time ag line_number
    ...
    ag line_number  0.68s user 1.03s system 459% cpu 0.371 total

0.37s vs 1.86s. You can see the output of both commands in my gist[1]. Ag obeys .gitignore/hgignore/etc by default, so it shows fewer results.[2] Since sift can't search files with greater than 2^18 characters between newlines, it prints out quite a few error messages. This typically happens with binary files and minified css/js.

You may be wondering why I added command line options to sift. They are to enable grouping and line numbers, making the output similar to ag. This may seem cosmetic, but it actually affects performance. Displaying line numbers means you have to count line numbers. That requires reading the entire matching file and adding up the newlines.

One last bit of benchmarking: If I tell ag to do an unrestricted search (meaning both ag and sift search the same files) and I tell sift to format its output like ag (meaning both count line numbers), the results are much closer. Sift takes 1.85 seconds. Ag takes 1.60 seconds. But unlike sift, ag doesn't bail on big files without newlines. It dutifully reports all the matching .o files.

Really though, sift and ag are trying to solve different problems. I designed ag to be a code searching tool. From what I can tell, sift is meant to be more of a grep replacement. Instead of worrying so much about which one is faster, use the right tool for the job.

1. https://gist.github.com/ggreer/96ed22bab57d1e791b9d

2. I consider this a feature. The goal of ag is to find what you're looking for, not to show everything remotely related to what you're looking for. If the default search doesn't find anything, you can quickly add the "-a" option.