Safe ways to do things in bash

1. This article contains excellent advice and should be starred for later retrieval.

2. Having basic scripting skills will make you a way better programmer. Many times I've done huge refactors and needle-in-hay-stack searches using only shell commands.

3. Shell is the universal language.

4. Bash isn't that bad once you get used to it (seriously. I'll grant you tho that arrays are still nasty ;-) ).

5. Bash is not that dangerous if you follow best practices. Don't be lazy!

6. You will not regret getting really good at shell script. You'll have to take my word for it now because you don't know what you're missing.

[1] https://github.com/koalaman/shellcheck

[2] https://github.com/SublimeLinter/SublimeLinter-shellcheck

> Should I use curly braces?

    Bad: some_command $arg1 $arg2 $arg3
    Extra bad (cargo culting unnecessary braces): some_command ${arg1} ${arg2} ${arg3}
    Correct: some_command "${arg1}" "${arg2}" "${arg3}"
    Better: some_command "$arg1" "$arg2" "$arg3"

> Shellharden will rewrite all these variants into the "better" form.

I prefer the "${bracey}" form for all variable usage. Yes it's marginally more verbose but it has the advantage of being consistent, easier on the eyes due to less overall quoting when part of full string interpolation[1], and cleaner diffs[2] as converting "${foo}" to "${foo}-bar" only leads to word-diff of "-bar".

[1]: "${foo} bar baz" v.s. "$foo"" bar baz"

[2]: You are source controlling your shell scripts right?

I really don't understand why bash still exists. I've switched to fish and am much happier with the change.

Nope. On the contrary, it says: Applications should note that the standard PATH to the shell cannot be assumed to be either /bin/sh or /usr/bin/sh

Source: http://pubs.opengroup.org/onlinepubs/009695399/utilities/sh....

A pedantically-compliant shell script should not have shebang at all.

if you are on busybox with ash none of this is helping (except shellcheck which is great).

- Relative unportability - Poor noncompliant sh interpreter - Poor performance (can be 4x slower than POSIX sh shells like dash for certain tasks)

I personally think bash is always the wrong choice. Use POSIX sh (or a real language). POSIX sh is 98% the same thing, there's no good reason to even use bash over sh in the vast majority of cases. It's just this blight that won't go away.

Given that bash is mostly sh anyway, most of this writeup applies to sh, too. AFAIK the only thing bash-specific here is arrays.

- The former just gives you a string whose contents are identical to that of var.

- The latter would do similar for the output of cmd, except that it strips away the trailing newline. This is often not an issue, but can be crucially important in some cases, and can catch you off-guard.

The point I'm making here is that it's actually quite difficult to get a string that literally has the contents you want. The fact that *nix lets you put pretty much any characters in file names (even newlines) means that, just like in Windows, your scripts can actually fail even when you try to quote things "properly".

Also, I prefer using [ condition ] for tests instead of the less-portable [[ cond ]] syntax despite the latter being more feature-rich. Didn’t see that one in there.

    IFS=$'\v' read -d '' -ra a < <(printf '%s\v' "$s")

    a = s.split("\v")

These are great tools to have, but don't blindly invoke them as magic spells if you don't actually understand the implications.

It proves the point that it's a gotcha but those examples seemed sensible to me. As far as I understand it, `set -e` doesn't turn every unchecked, non-zero exit code into an exception because there's no way of knowing whether the function or sub shell you're invoking was written by you or pulled in elsewhere, and as a result you don't know if a non-zero exit code is a legitimate, show-stopping failure.

Those functions and subshells might as well be mini inline executables and in that context it makes sense to only check the final output. If that's horribly wrong and confusing, maybe there should be a move to make `set -e` the default so all error handling is explicit, but you've got other languages for that that don't involve throwing `|| true` onto the end of every unimportant command you run.

I also realise that this doesn't make a case for Bash being intuitive. Precisely the opposite. But I suppose you have that with a shell where it's more important to be adaptive to the person behind the keyboard at the expense of the purity of the implementation. Especially considering the history of it all.

It's written by the same people who wrote the python ssh scripting framework fabric.

http://www.pyinvoke.org/

Works great!!

(Luckily it entered vim relatively soon, from where I could kill the process).

Things like being able to open a socket e.g. using the /dev/tcp/<ip>/<port> stuff give me the willies a bit.

I lean towards Ruby if it's going to be anything longer than a few lines, or requires anything but the simplest of logic/commands. Otherwise I was always told /bin/sh is likely to be the most portable, so tend to use that in the absence of any other good reason.

I do find that once my bash script is going over a hundred lines or so it's likely a good time to move to python.

I love bash, it's great; at some point setting up a proper script (in bash) with arguments, options, logging and or validation you end up spending more time getting it to work than you do on solving the actual problem; enter your favorite programming language here.

  $ cat - > foo
  #!/bin/bash
  a="BCD"
  [[ $a =~  .C.  ]] && echo 1
  [[ $a =~ ".C." ]] && echo 2
  ^D
  $ bash foo
  1
  $

  #!/usr/bin/env bash
  #--------------------------------------------
  # Default Bash Script Header
  set -eu
  trap stacktrace EXIT
  function stacktrace {
          if [ $? != 0 ]; then
                  echo -e "\nThe command '$BASH_COMMAND' triggerd a stacktrace:"
                  for i in $(seq 1 $((${#FUNCNAME[@]} - 2))); do j=$(($i+1)); echo -e "\t${BASH_SOURCE[$i]}: ${FUNCNAME[$i]}() called in ${BASH_SOURCE[$j]}:${BASH_LINENO[$i]}"; done
          fi
  }
  
  SCRIPT_DIR="$(dirname "$(readlink -f "$0")")"
  #--------------------------------------------

From my point of view, BASH will never disappear, because it's not a living thing that runs out of food or habitat and dies. That doesn't happen, unless there's a major revolution in computing that makes current paradigms obsolete, something on the scale of the disappearance of the dinosaurs.

Until that happens, I welcome any projects that aim to decrease the amount of buggy BASH in the wild. I avoid it as much as possible, but if someone's going to use it, at least they'll have safety nets to reduce the possible damage.

PS: I have a feeling my metaphors are all over the place, but I hope that doesn't detract from the message.

>Globbing is easier to use correctly than find.

For very simple purposes or small file trees, sure. Outside that: find is incredibly more powerful, and worth using in many cases. If nothing else, learning to use `find . -path ignore_this_path -prune -o *.ext -print` can change e.g. a Go project script from visibly-slow to instant. (e.g. the latest place I used this went from 1-5+ (hot vs cold) seconds to 5-50ms)

For example:

    parent_dir = "$(basename $dir)"

That's what I dislike about Bash.

Also, always start your scripts with

    set -e

Also, I always make mistakes when using [ and [[.

The result of a variable substitution isn't subject to wilcard expansion, whether quoted or not.

If your only reason to quote "$foo" is because you think foo expands to a globbing pattern, and no other reason is justified, you can drop the quotes.

This is a dangerous advice. set -u indeed requires more verbose array handling in Bash < 4.4, but it also catches code like this:

  rm -rf "${OOPS_UNDEFINED}/"

One day I'll learn the Ruby way of doing shell one-liners and that'll hopefully keep me out of man pages for that sort of thing forever.

It's not published to crates.io, and doesn't have a license or Cargo.toml

This advice only works in bash 4.4, and many common distros are on bash 4.3, like Ubuntu 16.02 LTS. (bash 4.4 was released September 2016.) Because of this bug, the section "how to begin a bash script" is version-specific and awkward IMO.

If you need to process untrusted filenames, use arrays, but otherwise it's probably more trouble than it's worth. [1]

I want my scripts to work on older versions of bash, and I think 'set -u' is important, so I mostly get by without arrays. It's not ideal, but shell is full of compromises.

A workaround is to use ${a[@]+"${a[@]}"}, which avoids the bug in bash 4.3, but that seems too ugly to recommend.

More details in this comment I wrote on the same article: https://lobste.rs/s/4jegyk/how_do_things_safely_bash#c_kmldw...

[1] Thirteen Incorrect Ways and Two Awkward Ways to Use Arrays http://www.oilshell.org/blog/2016/11/06.html

http://xon.sh/

https://github.com/neurobin/shc

please don't down vote me

Clarity is king people

    if [ ${#array[@]} -gt 0 ]; then

    if (( ${#array[@]} > 0 )); then

If you need safety don't use Bash.

Command substitutions also come in this form:

    Correct: "`cmd`"
    Bad: `cmd`

This is how one can tell a rookie just learning to program in shell: usage of $() syntax is limited to Bourne family of shells which implement that particular POSIX specification aspect.

Backticks, on the other hand, although they incur a performance penalty since they spawn a subshell, make one's code instantly portable across all UNIX-like operating systems and even across disparate shell families, as they work exactly the same in C-shells. (Whether one should program in a C-shell family is a different discussion.)

The subshell performance penalty is negligible in 99% of the cases as this 1970's technology has tiny memory and processor overhead due to the fact that it's been developed on systems with small memory and a slow CPU, so it had been optimized for performance.

Over my 30+ years of shell programming, I know of only one documented instance where the $() construct which doesn't spawn a subshell made a difference, and was the only time it was actually a valid requirement:

https://www.joyent.com/blog/building-packages-at-scale

but even then, the author ended up using dash, not bash.

For maximum portability and closest adherence to POSIX, program in Korn shell, ksh93. (Modern versions of ksh implement ksh93 functionality.) Then you may safely use $() and be assured it will work in all Korn shells across different operating systems (even in ksh88).

Otherwise, DON'T avoid using backticks, because you will be giving away portability for no good reason. Don't program in bash, but in original Bourne shell (sh) for maximum portability across different operating systems; don't assume that you can use bash constructs in Bourne shell (as /bin/sh on GNU/Linux tells bash to run in Bourne shell emulation mode, but that mode isn't implemented completely or correctly, since bash constructs are still accepted). Always test your shell code on a traditional UNIX like HP-UX or a Solaris derivative like SmartOS if possible, with a real Bourne shell.