Hacker News Reader

Comment by lukeschlather | Hacker News Reader

The result of `cat a b > a` couldn't possibly be the result that the user intends, so it seems like poor design by definition. I can see how it would be tricky to handle it properly, but I do think it's possible to handle it properly without compromising the expressiveness of the shell.

View on HN

In modern bash, this case is already protected with the "noclobber" option (enabled with "set -C" or "set -o noclobber"). It is described in the "REDIRECTION" section of bash(1), subsection "Redirecting Output".

It fails ">" redirections with an error message when it would clobber an existing file (you probably wanted to append to the file wiht ">>"). If you really intended this behavior, the new redirection operator '>|' has the old behavior.

    $ set -o noclobber
    $ echo foo > bar
    $ cat bar
    foo
    $ cat bar > bar
    bash: bar: cannot overwrite existing file
    $ echo baz > bar
    bash: bar: cannot overwrite existing file
    $ echo baz >| bar
    $ cat bar
    baz

edit: fixed typo

That's a nice feature, but it should definitely be turned on as a default rather than hidden behind an option.

That would break a very large number of existing scripts that depend on being able to replace a file with a traditional ">" redirection.

You can set it as the default for your own interactive shell if you want by adding

    set -o noclobber

to your ~/.bashrc file. It is probably a good idea to add it after the line that check if it is being used non-interactively, which probably looks something like this

    if [[ $- != *i* ]] ; then
        # Shell is non-interactive
        return
    fi
    # add interactive-shell-only settings here

While you're editing .bashrc, you may want to also add these options that are available in modern bash that I find make the experience a lot nicer.

    # support new glob patterns like !(foo|bar|...)
    # for example, this move everything EXCPET the .h files
    #     mv !(*.h) /some/dir/
    shopt -s extglob

    # support ** in globs to match zero-or-more-recursive-directories
    shopt -s globstar

    # if you're annoyed at having to type "nohup" to prevent
    # programs from closing when you close the shell window
    shopt -u huponexit

Right, it definitely would break a lot of existing scripts, so I understand why its not a default. The thing is though, that really proves the argument that some parts of Unix have a bad out of the box user experience! I personally love Unix and find it amazingly powerful, but I've made the same mistakes as everyone and killed entire boxes, and really with some simple UX changes that could be fixed in the bulk of cases.

Edit: I forgot to add, thanks for the detailed answer!

noclobber is also an option setting in zsh, and I always have it set. The little extra confirmation it requires before clobbering is actually a nice confirmation that you are doing what you intended when you did intend it, and a nice rescue in case you didn't.

It won't save you from clobbering from another machine when using scp, though, so watch out.

Nice feature, never heard of that before. Thanks!

At first I was going to say that handling this properly would require shell to have magic powers. After trying to come up with a proof of why it's impossible, I think I found a simple solution. There are two problems the author was complaining about. The first one is:

cat paper.* > paper.all

This has the potential to use up the whole disk. The fix here is easy: do globbing before redirection. But this does not solve the other problem:

cat a b > a

This has a simple workaround which is even easier to type: cat a >> b. But the trap remains. How could the shell fix it? By pretending the user wrote this instead:

cat a b > temp; mv temp a

Let's look at the trade offs. We do a bit of extra work: one rename operation. This should be pretty cheap, and solves this problem as well as paper.all.

The real trouble I think is coming up with a good name and location for tmpfile that is guaranteed not to mess up the operation of the command we're running. If we put it in the current directory, we get the wrong answer from ls. If we put it in /tmp, it will mess things up when we're working in /tmp. However, after doing a few minutes of research, it appears that this problem is already solvable on Linux: by using the O_TMPFILE flag, there is no need to give the file a name at all, until after the command finishes running.

Sometimes I start a long-running process, redirecting it's output, and periodically look at the output file while this process is running. Your proposed solution wouldn't allow for that.

It would be possible, but not convenient: /proc/pid/fd/1. Ick. Ok, maybe the cure is worse than the disease. I'm out of clever ideas for now.

What if 'a' is a device?

Ouch. Don't do that for devices. Or named pipes I guess. This idea has more problems than I thought at first.

> The result of `cat a b > a` couldn't possibly be the result that the user intends

The result of cat a b > a is precisely the result that the user intends, provided that the user has actually read the documentation.

If the user knew what they were doing they wouldn't have typed that. `cat a b > a` is the same as `cat b > a`. There's no reason to add the a before b except to confuse.

It's in the context of why cat paper* > paper.all blows everything up. Of course it's meaningless.

Reply on news.ycombinator.com