Hacker News Reader

Comment by Xophmeister | Hacker News Reader

A “better” fix would be:

    find . -type f -exec grep -H foo {} +

View on HN

Yeah, find ¦ xargs is definitely an antipattern, but in practice I use it all the time. Either because I forget the syntax for find -exec, or because I'm feeling principled that find -exec breaks the Unix philosophy of doing one thing well and being composable, or because my script started using ls and got changed to use find.

xargs is also more efficient than -exec, as it stuffs multiple files to the target command. Esp. for rm, instead of executing a copy for each file.

doesn't

    -exec <command> {} \+

do that too? (and we're back at the "remembering specific syntax" point: knowing xargs helps you everywhere)

Both of these are valid:

  -exec <command> <args> {} +
  -exec <command> <args> {} <args> \;

(The backslash is unnecessary for +)

The first one, the one used here, acts like xargs. The second one executes <command> individually for each file found. Because the "+" version continually adds files to it, it must be last, but the "\;" version doesn't have that restriction since it's a single substitution. Just to be clear with an example:

  $ ls
  one  three  two
  $ find . -exec echo 11 {} 22 \;
  11 . 22
  11 ./two 22
  11 ./three 22
  11 ./one 22
  $ find . -exec echo 11 {} 22 +
  find: missing argument to `-exec'
  $ find . -exec echo 11 {} +
  11 . ./two ./three ./one

Most people seem to only learn one of them, even getting them mixed up. For example, I had learned the "\;" one and for years thought they both acted like that, with "+" simply being an alternate end-of-command identifier to avoid needing to escape the semicolon.

The “+” terminator to -exec has the same effect for find.

Why is find | grep an anti-pattern? Is eschewing memorizing the flags of one particular command in favor of using a pipeline which is much more broadly applicable really an anti-pattern?

Heck, I’m at the point now where I pipe tar into gzip.

I assume you mean “find | xargs”, rather than “find | grep”; the latter would grep against the paths that find matches, rather than their contents. Anyway, the reason “find | xargs” is considered an antipattern is because xargs splits its input on newlines and this would break on any paths that contained a newline. Arguably, you can get around this by using find’s -print0 and xargs’s -0 arguments, respectively, as paths cannot contain null bytes, but you might as well have used -exec at this point!

I think there is a cottage industry around telling people that they're using the interactive shell wrong. This all started with "useless use of cat", which people still bitch about on Reddit and Twitter on a regular basis, and so whenever someone finds something a little weird about UNIX, they are quick to call a person asking a question about it dumb. I think it might be stockholm syndrome, or a symptom of one's greatest achievement in life thusfar being reading the man page for find. Or maybe they genuinely think they're helping. I dunno.

Ultimately, it's a cascading failure of bad design. Good design is that find and xargs each do one thing and do it well. Find can output files to anything! Xargs can xargify input from anything! Bad design is that find uses \n as the output record separator and that xargs uses \n as the input record separator, but \n can appear in filenames! This design can never work and will always lead to difficult-to-debug problems. So instead of fixing it, we blame the user for not knowing "oh, well find has xargs built in. sort of. a lot of the features are there. not all of them. but some. so never use find | args, smile." or "well, we designed the unix shell to be easy to use... but there is this corner case that we knew about and could have protected you from, but we chose not to, so 1% of the time you'll rm -rf / with your xargs command, UNLESS YOU REMEMBER THIS ONE WEIRD TRICK which is to use -0 to use an out-of-band symbol to separate records which we could have just made happen by default but chose not to, just to mess you up!"

I appreciate the efforts of people that are re-imagining the interactive shell, like PowerShell. I am not very productive with PowerShell (to get a listing of files in your directory, just type "Could-We-Make-It-Even-More-Verbose -PerhapsInTheNextVersionWeWill!") but it's a good prototype of a good shell. I have personally stepped on enough UNIX landmines to be generally OK with the compromises, or more typically don't write shell scripts for anything that will be run more than twice. It's bad though, and I don't think the users are to blame.

> Bad design is that find uses \n as the output record separator and that xargs uses \n as the input record separator, but \n can appear in filenames! This design can never work and will always lead to difficult-to-debug problems. So instead of fixing it

Perhaps, but it also can't be avoided. Everything can appear in filenames. We want filenames to be able to contain any character.

The most high-profile attempt to "fix" the "problem" you identify here is of course the ASCII standard, which defines several separators (0x1C - 0x1F) just for the purpose of delimiting fields with bytes that can't appear in data. Except of course nobody cares -- the solution was stillborn -- because data can contain any bytes that anyone wants it to.

> I am not very productive with PowerShell (to get a listing of files in your directory, just type "Could-We-Make-It-Even-More-Verbose -PerhapsInTheNextVersionWeWill!")

Well, you could do this by typing Get-ChildItem, but it would be both easier and more standard to type one of "gci" (what Microsoft would like you to use), "ls" (meant for those used to unix) or "dir" (for those used to dos).

> Perhaps, but it also can't be avoided. Everything can appear in filenames. We want filenames to be able to contain any character.

Who is "we" ? I wouldn't mind some 'sane' limits, eg UTF8, no control characters and no names starting with a dash.

Still hoping for the day that some distribution sets a mount option to enforce sane filenames by default and starts weeding out the application bugs it'll trigger..

https://dwheeler.com/essays/fixing-unix-linux-filenames.html

There are two options. \0 can't appear in filenames, which is certainly questionable (why is that byte special?), so using \0 as the separator works. A better way of solving the problem is to just prefix the record with a field that represents the length. People are allergic to the length prefix because, say you set it to 2 bytes, now your max file length is 512 characters AND you have two bytes of overhead for every filename. Whereas using \0 means that you have only one byte of overhead and can have infinitely long filenames.

Interestingly, I guess nobody has ever died because of a buffer overflow. Therac-25 was integer overflow and bad synchronization (and an open loop control system, which people still love). So I guess it doesn't matter.

> just type "Could-We-Make-It-Even-More-Verbose -PerhapsInTheNextVersionWeWill!"

PowerShell can be as terse as bash. Just use the aliases (which are actually real aliases in PowerShell), leverage default parameters and shorten parameter names (or use their aliases) to just use enough character to disambiguate.

To get a file listing of your directory, simply type

ls

If you want to know why this works, type

    help ls

and PowerShell will answer you with the help on "Get-ChildItem".

You could also type this to explain the `ls` command:

    gcm ls

And PowerShell will answer

    CommandType     Name                                               Version    Source
    -----------     ----                                               -------    ------
    Alias           ls -> Get-ChildItem

`gcm` is itself an alias for Get-Command - a command that gets information about a command. Executing `gcm gcm` will produce this output:

    CommandType     Name                                               Version    Source
    -----------     ----                                               -------    ------
    Alias           gcm -> Get-Command

Incidentally, why does `find` have a `--delete` option in the first place? That's not very do one thing only and do it well. `find` should search/find and do that well. Why is it also a file-deleter?

Going back to the example from TFA, in PowerShell you would delete files not matching a pattern using this command:

    ls -ex *.py -rec -file | rm

Or, if you do want it verbose, using full names instead of aliases and shortened parameter names:

    Get-ChildItem -Exclude *.py -Recurse -File | Remove-Item

Reply on news.ycombinator.com