Hacker News Reader

Comment by vore | Hacker News Reader

I think bash has a lot of gotchas due to its original purpose, and it's definitely more error-prone to write bash scripts than to e.g. write Python scripts, so that should be a consideration.

Ideally we should all try to more programming languages that are harder for us to shoot ourselves in the foot with, rather than easier ;-)

View on HN

I will just leave this here.

"I am a strong believer that Bourne-derived languages are extremely bad, on the same order of badness as Perl, for programming, and consider programming sh for any purpose other than as a super-portable, lowest-common-denominator platform for build or bootstrap scripts and the like, as an extremely misguided endeavor. As such you won’t see me spending many words on extensions particular to ksh, Bash, or whatever other shells may be popular."

http://www.etalabs.net/sh_tricks.html

I think it's easy to sort of gloss over how good shell scripts are at the sort of things they actually are good at.

Consider this:

  for f in *png; do convert $f $(echo $f | sed 's/png$/webp/'); done

This is like at least a screen of code in in most languages, you get that sinking feeling where you need to fork a process and deal with its lifecycle and deal with some standard library to list files and don't forget to close the file handle for the directory and god forgive if you run a program with actual output then you need to read from a pipe. Like, geesh, do I have to write a whole program to convert a few files?

In a shell, it's just something you type up because you needed to convert a few files. Don't get me wrong, shell scripts are extremely bad at most things involving non-trivial branches, long-lived state, really most things other languages are good at, but man are they ever good at some things most programming languages are bad at.

    lee@sp3tmp1:ls *.png
    'a b.png'   c.png
    lee@sp3tmp1:for f in *png; do convert $f $(echo $f | sed 's/png$/web'); done
    convert-im6.q16: unable to open image `a': No such file or directory @ error/blob.c/OpenBlob/2924.
    convert-im6.q16: no decode delegate for this image format `' @ error/constitute.c/ReadImage/575.
    convert-im6.q16: unable to open image `b.png': No such file or directory @ error/blob.c/OpenBlob/2924.
    convert-im6.q16: unable to open image `b.png': No such file or directory @ error/blob.c/OpenBlob/2924.
    convert-im6.q16: no decode delegate for this image format `' @ error/constitute.c/ReadImage/575.
    convert-im6.q16: no images defined `b.webp' @ error/convert.c/ConvertImageCommand/3229.

[etc.]

I think you could be making a more complicated point.

Better:

    for f in *.png; do convert "$f" "${f/%.png/.webp}" ; done

Quoting the arguments lets it correctly handle file names with funny characters, including spaces. Using the `${parameter/pattern/string}` parameter substitution means the shell does all the work itself without invoking an external command (other than convert, of course). The '%' in the parameter substitution requires the pattern to match at the end, so "foo.png.png" becomes "foo.png.webp" and not "foo.webp.png" -- an unlikely corner case, but you might as well do it right. ("foo.png.blah" doesn't match "*.png" in the first place.)

For elaborate commands like this, I typically run it once with an "echo" inserted ("echo convert ..."), confirm that the resulting commands make sense, and then hit up-arrow to edit the command and delete the echo.

https://www.gnu.org/software/bash/manual/html_node/Shell-Par...

Quoting is not enough, as that won't handle filenames beginning with dash correctly.

Your script fails if the files referenced by $f contain white spaces or a newline.

You should script defensively, for when this is run on Windows or elsewhere [maybe Android] that will break this assumption.

Newlines can also be part of a filename. For extreme rigor, use find -print0 and xargs -0.

I have covered some of this here:

https://www.linuxjournal.com/content/parallel-shells-xargs-u...

(Somebody took me to the Hacker News front page for this, and nobody said that I made a mistake.)

I agree. It also fails if the filename starts with a dash.

> You should script defensively, for when this is run on Windows or elsewhere [maybe Android] that will break this assumption.

I think you are missing the point. I wouldn't run a shellscript on Windows or any other place than the terminal I entered it into. It's an expression of something I want to do at that particular time in that particular directory.

The entire point of my post is that if I wished to write something portable and robust, I wouldn't use bash in the first place. That's not what bash is good at. But quick one-offs, holy crap is it good at that.

I've run many Borne/POSIX shell scripts on Windows for things that I've needed to do.

The most straightforward POSIX shell on Windows is the Busybox port, which uses the Almquist shell (ash) with many bashisms reintroduced.

https://frippery.org/busybox/

> fails if the files referenced by $f contain white spaces or a newline.

This is said every time Bash scripting is mentioned. Why people can't get into the habit of quoting all variables in Bash is beyond me and the mention of white space makes sense; but how often, if ever, have you encountered a file name that contained a newline?

Most of the Unix tools are line oriented by default anyway and are frequently used to manipulate lists of file names and paths.

It's safe to omit quotes for POSIX shell variables that are initialized as numeric, and you enforce to remain numeric.

All else should be quoted, and newlines can happen, primarily from abuse. Do not trust user input!

Calling fopen() on a file with every ASCII character will work on POSIX, as long as it does not contain a null or forward slash (/). Coding defensively means that this does not crash your processing.

I'm not objecting to quoting; I do it even when the variable contains, or should contain, a number.

I was objecting to the knee jerk inclusion of warnings against file names including newlines. Has anyone encountered one in the wild that was not itself an error?

>> I think it's easy to sort of gloss over how good shell scripts are at the sort of things they actually are good at.

>> Consider this:

> for f in *png; do convert $f $(echo $f | sed 's/png$/webp/'); done

This is buggy, and it actually represents exactly the problem with Bash scripting.

I think considering the purpose of the language, it does rather well. It's really easy to spawn and pipe processes, far more so than in almost any alternatives. Like it's so easy that it's easy to forget that this is like a 5-10 line overhead dance to launch and read from a process in most languages.

The problem usually come when you attempt to use it for nontrivial programming. But that just not what it's for, any more than the difficulty in building an OS kernel is a legitimate complaint against PHP.

I think this is what the post you were replying to is saying.

Stephen Borne famously disliked C, and much preferred Algol.

This is important to know in the context of what you are saying.

The Algol structures in the Borne shell are the most incongruous parts of UNIX.

https://research.swtch.com/shmacro

By that logic shouldn’t we also stop using programming languages that aren’t threadsafe? Segfaulting could be prevented if C++ wasn’t so esoteric.

Reply on news.ycombinator.com