Comment by tieTYT | Hacker News Reader

Comment by tieTYT | Hacker News Reader

Parent Full thread

I wrote my first shell script that actually matters a few weeks ago (in other words, I'm a total newb). Its purpose is to validate the configuration of another app. It seems to me that "set -o pipefail" would get in my way, because I'm often doing things like this:

    adapter_properties_home_community_id=$(cat ${EXTERNAL_VARIABLE}/adapter.properties \
    | grep -P '^HomeCommunityId=' | sed 's/HomeCommunityId=//g' \
    || err "line $LINENO: Could not find HomeCommunityId in the adaper.properties")

If I "set -o pipefail", won't any failure in this chain of pipes halt the program? That will prevent the user from seeing the useful error message that says "Could not find HomeCommunityId in the adaper.properties"

I'm also concerned about "set -u". That EXTERNAL_VARIABLE is set external to my validator script and available to the shell. Any appliance with the app should have that variable set. I have a check to make sure it's set and it's valid, but if I had "set -u", wouldn't that prevent me from checking its value because it hasn't been defined in the script? What's the work around to this, because I see how "set -u" would be very useful in most other situations.

Some specific remarks on your shell script at the end...

> If I "set -o pipefail", won't any failure in this chain of pipes halt the program?

No, due to your use of the || operator:

    foo | bar || baz

...as a whole will always exit 0, even if foo or baz exits non-zero. The precedence there is actually

    ( foo | bar ) || baz

I.e, the pipe operator binds tighter than any of the logic operators. Also, `set -o pipefail` doesn't cause your shell script to exit on the first error; you need `set -e` or explicit exit status checking for that. What `set -o pipefail` actually does is change the way exit status for a pipeline as a whole works. Consider the following pipeline:

    foo | bar | baz

Without `set -o pipefail`, if either foo or bar exit non-zero, but baz exits zero, this pipeline will appear to have succeeded, because by default a pipeline gets the exit status of its last element. Ignoring errors is bad. So `set -o pipefail` makes the pipeline take the exit status of the last element that failed (if any failed).

Regarding `set -u`, I don't personally use it, I just check that kind of stuff at the beginning of the script.

Now, on to your shell script example: in practice you'll never see an error because both grep and sed return true even when no matching or substitution is done. You also have a useless use of cat [1] at the beginning; just use grep directly on that file.

I would suggest simply doing

    sed -ne 's/^HomeCommunityId=//p' "${EXTERNAL_VARIABLE}"/adapter.properties

for grabbing that variable. Then see if the variable is empty with a `-z` test.

Hope this helps. Shell is weird but definitely worth learning!

[1] http://en.wikipedia.org/wiki/Cat_(Unix)#Useless_use_of_cat

I should have mentioned that

  1. I am already using set -e
  2. That `err` function has an `exit 1` as its last step, 
  which forces the program to stop running when combined with the first point.

> Now, on to your shell script example: in practice you'll never see an error because both grep and sed return true even when no matching or substitution is done.

That's not what I'm seeing:

    > echo "hi" | grep "bye"
    > echo $?
    1

I'm typing that into my shell.

Thanks for the reply, it was very helpful.

Ah, you're correct about this:

    echo "hi" | grep "bye" ; echo $?
    1

Reply on news.ycombinator.com