Hacker News Reader

Comment by Scarbutt | Hacker News Reader

The infamous "useless use of cat"

It annoys me whenever somebody brings this up. People do this because it's easier to modify commands at the end of your prompt than near the beginning. Especially in this case, where the grandparent wants to quickly switch between benchmarking grep, ripgrep etc.

Not useless at all.

Though a neat thing to know is that redirections can appear anywhere on the line in Bourneish shells:

    time < title_complete_json.txt grep tt9184994

As it happens, when benchmarking grep-like tools on large files, this is actually somewhat likely to make a significant difference since the "useless" use of cat actually forces the file to be read sequentially as a stream (because it's piped through), while a process receiving a file on standard input can treat it as a file and e.g. mmap it.

I found a benchmark that suggests the performance gap between a pipe and input redirection is insignificant for most workloads, but there would indeed start to be a difference for the exact case discussed here (grepping a large file): http://oletange.blogspot.com/2013/10/useless-use-of-cat.html

Thanks for your tip, it's clearly the best of both worlds for this purpose.

Incidentally, ripgrep will actually be faster in this case generally (on Linux at least) when given an explicit file path, since that lets it use memory maps. For example:

    $ time rg QZQZQZQZ < subtitles.en.txt

    real    1.842
    user    0.552
    sys     1.287
    maxmem  9 MB
    faults  0

    $ time rg QZQZQZQZ subtitles.en.txt

    real    1.210
    user    0.776
    sys     0.433
    maxmem  9510 MB
    faults  0

(See my other comments in this thread for where to get subtitles.en.txt.)

To clarify, it would have been prettier (conceptually less clunky and shorter) to run this:

time grep tt9184994 < title_complete_json.txt >> result

grep takes a file directly, so this could simply be:

time grep tt9184994 title_complete_json.txt >> result

That's relevant in some scripts, where unnecessary forks might actually add up to some meaningful resource usage. Not relevant when typed at a tty.

Surely using cat like this is more in line with the unix philosophy?

Reply on news.ycombinator.com