Writing Robust Bash Shell Scripts | Hacker News Reader

Writing Robust Bash Shell Scripts

davidpashley.com

51 pointsFeanim21 comments

I can't read the article now, but in my opinion the fact that it's so difficult to write robust shell scripts above a very basic level of complexity (e.g. do this, then do that, then do that, then quit) really means you should be scripting in perl/python/ruby/etc instead.

Yes, the overhead for anything quick and dirty is higher, but if you're writing libraries, you only have to write them once.

The only real benefit to writing something complex entirely in a shell script is if you can't control the environment it will run in, and you need it to run everywhere, and you can't make users install an interpreter, which are increasingly rare requirements, and even then you have to write it in `sh` anyway.

These are generally good tips. I swear by

    set -e # same as set -o errexit

and I like using list constructs (&& ||) in lieu of simple `if` test blocks but it's really important to remember that list constructs aren't exactly the same. This always catches me out:

    if [ -e "missingfile" ]
    then
        echo "you'll never see this"
    fi

(above runs fine with set -e)

    [ -e "missingfile" ] && echo "you'll never see this"

(while that one correctly fails and thus `set -e` fails the script)

I generally stick an `|| true` on the end of those list constructs.

fwiw.

Text only cache, courtesy Google - http://webcache.googleusercontent.com/search?q=cache:-aFhMx4...

He suggests using

    if [ "$filename" = "foo" ];

Why not

    if [[ $filename == "foo" ]];

EDIT: aware of the fact that [[ ]] is a bash extension, but the actual title of the post is "Writing Robust Bash Shell Scripts", implying that it should be using the best stuff for BASH.

    $ cat foo.sh
    #!/bin/bash
    echo $BASH_VERSION
    unset filename
    if [[ $filename == "foo" ]];
    $ ./foo.sh
    4.1.2(1)-release
    ./foo.sh: line 5: syntax error: unexpected end of file

If $filename evaluates to an empty string, then the last line of code becomes

    if [[ == "foo" ]];

Similarly, if $filename evaluates to a b c, then the last line of code becomes

    if [[ a b c == "foo" ]];

The bash interpreter has problems with both of these.

No, that is just wrong. In double-brackets, bash will handle expansion and testing of unset variables and variables with spaces just fine, because word splitting and pathname expansion are not performed in double braces -- they are shell syntax, as opposed to single brackets, which are commands. Your syntax error happened because you don't have a fi matching that if.

As a counterexample:

  (echo $BASH_VERSION; 
  foo=foo; [[ $foo = foo ]] && echo $foo; 
  foo="1 2 3"; [[ $foo = "1 2 3" ]] && echo $foo; 
  foo=''; echo ${foo?"No foo for you"}; 
  unset foo; echo ${foo?"No foo for you"})
  4.2.10(1)-release
  foo
  1 2 3
  
  bash: foo: No foo for you

That last one is one of the few ways you can tell whether a variable is unset or merely set to an empty string.

lrn2bash.

First is Posix, the second is bash. The first is more portable.

The title of the article is "Writing Robust Bash Shell Scripts", which differs from the title of the HN post.

    if [[ $filename == foo ]];

Which is the correct idiomatic way. The article's author is wrong in enclosing foo in quotes.

  set -u

Another great thing I could have read in the effin manual.

He could have added a little reminder about the ${VAR:-default} syntax. [[ http://www.ooblick.com/text/sh/ search for Quasi-variable constructs ]]

Great read, thanks.

Or look at the manpage

       ${parameter:-word}
              Use  Default  Values.  If parameter is unset or null, the expan-
              sion of word is substituted.  Otherwise, the value of  parameter
              is substituted.

Agreed, I was just sayin for the sake of the article, he brings up the subject of unitialized variables, without mentioning this other way.

The locking section could also be dealt with by using mkdir for locking. mkdir is atomic. (Note that mkdir -p is not.)

This is not always true on NFS for various reasons too stupid to go into. There are precious few things so stupid that someone has not committed them to an NFS server somewhere, but the retry semantics in the protocol make it generally awkward and it's possible for two mkdirs to both succeed or both fail but in both cases for the directory to exist. See http://www.mail-archive.com/freebsd-hackers@freebsd.org/msg2... and http://stackoverflow.com/questions/185451/quick-and-dirty-wa... . The only things NFS guarantees is that symlink, rename, and open with the O_EXCL flag (and that only in v3), will be atomic.

It's strange how the quoting trick seems to work, even when double quotes appear in in a variable. For example:

    Ives ~ $ FN='"a b"'
    Ives ~ $ if [ "$FN" = '"a b"' ]; then echo "Does not matter"; fi
        Does not matter

Still works fine, so clearly something more than simple variable expansion is going on here. It seems like the end-of-literal-string matching is performed before variable expansion.

The premise is flawed. Robustness requires a shell that works the same across platforms and across time, and that means /bin/sh.

Most of the people reading this article, IME, are not familiar with /bin/sh, or its differences from /{bin,usr/bin,usr/local/bin}/bash, much less know how or why it is important to write robust AND portable shell scripts.

Why did you change the title of the post? The actual title according to the HTML is "Writing Robust Bash Shell Scripts" and your title is "Writing robust shell scripts"/

The author changed it, as you can see the url still has the old title

edit: changed

Ah, stealth edit

Did you pull this from my google+ post?

Reply on news.ycombinator.com