Hacker News Reader

Comment by chasil | Hacker News Reader

I will just leave this here.

"I am a strong believer that Bourne-derived languages are extremely bad, on the same order of badness as Perl, for programming, and consider programming sh for any purpose other than as a super-portable, lowest-common-denominator platform for build or bootstrap scripts and the like, as an extremely misguided endeavor. As such you won’t see me spending many words on extensions particular to ksh, Bash, or whatever other shells may be popular."

http://www.etalabs.net/sh_tricks.html

View on HN

I think it's easy to sort of gloss over how good shell scripts are at the sort of things they actually are good at.

Consider this:

  for f in *png; do convert $f $(echo $f | sed 's/png$/webp/'); done

This is like at least a screen of code in in most languages, you get that sinking feeling where you need to fork a process and deal with its lifecycle and deal with some standard library to list files and don't forget to close the file handle for the directory and god forgive if you run a program with actual output then you need to read from a pipe. Like, geesh, do I have to write a whole program to convert a few files?

In a shell, it's just something you type up because you needed to convert a few files. Don't get me wrong, shell scripts are extremely bad at most things involving non-trivial branches, long-lived state, really most things other languages are good at, but man are they ever good at some things most programming languages are bad at.

    lee@sp3tmp1:ls *.png
    'a b.png'   c.png
    lee@sp3tmp1:for f in *png; do convert $f $(echo $f | sed 's/png$/web'); done
    convert-im6.q16: unable to open image `a': No such file or directory @ error/blob.c/OpenBlob/2924.
    convert-im6.q16: no decode delegate for this image format `' @ error/constitute.c/ReadImage/575.
    convert-im6.q16: unable to open image `b.png': No such file or directory @ error/blob.c/OpenBlob/2924.
    convert-im6.q16: unable to open image `b.png': No such file or directory @ error/blob.c/OpenBlob/2924.
    convert-im6.q16: no decode delegate for this image format `' @ error/constitute.c/ReadImage/575.
    convert-im6.q16: no images defined `b.webp' @ error/convert.c/ConvertImageCommand/3229.

[etc.]

I think you could be making a more complicated point.

Better:

    for f in *.png; do convert "$f" "${f/%.png/.webp}" ; done

Quoting the arguments lets it correctly handle file names with funny characters, including spaces. Using the `${parameter/pattern/string}` parameter substitution means the shell does all the work itself without invoking an external command (other than convert, of course). The '%' in the parameter substitution requires the pattern to match at the end, so "foo.png.png" becomes "foo.png.webp" and not "foo.webp.png" -- an unlikely corner case, but you might as well do it right. ("foo.png.blah" doesn't match "*.png" in the first place.)

For elaborate commands like this, I typically run it once with an "echo" inserted ("echo convert ..."), confirm that the resulting commands make sense, and then hit up-arrow to edit the command and delete the echo.

https://www.gnu.org/software/bash/manual/html_node/Shell-Par...

Quoting is not enough, as that won't handle filenames beginning with dash correctly.

Your script fails if the files referenced by $f contain white spaces or a newline.

You should script defensively, for when this is run on Windows or elsewhere [maybe Android] that will break this assumption.

Newlines can also be part of a filename. For extreme rigor, use find -print0 and xargs -0.

I have covered some of this here:

https://www.linuxjournal.com/content/parallel-shells-xargs-u...

(Somebody took me to the Hacker News front page for this, and nobody said that I made a mistake.)

I agree. It also fails if the filename starts with a dash.

> You should script defensively, for when this is run on Windows or elsewhere [maybe Android] that will break this assumption.

I think you are missing the point. I wouldn't run a shellscript on Windows or any other place than the terminal I entered it into. It's an expression of something I want to do at that particular time in that particular directory.

The entire point of my post is that if I wished to write something portable and robust, I wouldn't use bash in the first place. That's not what bash is good at. But quick one-offs, holy crap is it good at that.

I've run many Borne/POSIX shell scripts on Windows for things that I've needed to do.

The most straightforward POSIX shell on Windows is the Busybox port, which uses the Almquist shell (ash) with many bashisms reintroduced.

https://frippery.org/busybox/

> fails if the files referenced by $f contain white spaces or a newline.

This is said every time Bash scripting is mentioned. Why people can't get into the habit of quoting all variables in Bash is beyond me and the mention of white space makes sense; but how often, if ever, have you encountered a file name that contained a newline?

Most of the Unix tools are line oriented by default anyway and are frequently used to manipulate lists of file names and paths.

It's safe to omit quotes for POSIX shell variables that are initialized as numeric, and you enforce to remain numeric.

All else should be quoted, and newlines can happen, primarily from abuse. Do not trust user input!

Calling fopen() on a file with every ASCII character will work on POSIX, as long as it does not contain a null or forward slash (/). Coding defensively means that this does not crash your processing.

I'm not objecting to quoting; I do it even when the variable contains, or should contain, a number.

I was objecting to the knee jerk inclusion of warnings against file names including newlines. Has anyone encountered one in the wild that was not itself an error?

>> I think it's easy to sort of gloss over how good shell scripts are at the sort of things they actually are good at.

>> Consider this:

> for f in *png; do convert $f $(echo $f | sed 's/png$/webp/'); done

This is buggy, and it actually represents exactly the problem with Bash scripting.

Reply on news.ycombinator.com