Hacker News Reader

Comment by bambax | Hacker News Reader

Yes. Also, maybe not have a delete action in the middle of a script. It's usually better to build a list of items to be deleted. In that case, two lists: items to be deleted, items to be kept. Then compare the lists:

- make sure the sum of their lengths == number of total current items

- make sure items_to_be_kept.length != 0

- make sure no two items appear in both lists

- check some items chosen at random to see if they were sorted in the correct list

At this point the only possible mistake left is to confuse the lists and send the "to_be_kept" one to the delete script; a dry run of the delete list can be in order.

View on HN

This. The original approach can fail horribly if there's a problem on the server when you run the script for real. Your code can be perfect but that's no guarantee the server will always return what it ought to.

I've had good success with this approach, have two distinct scripts generate the two lists, then in addition to your items here also checking that every item appears in one of the lists.

What do you recommend, to not get intro trouble if there are spaces or newlines in the file names?

Don't use a shell script.

Do you mean, always pass the list directly to the next script via function calls, without writing it to an intermediate file / pipeline?

Yes, use the list argument to Python’s subprocess.run for example. It’s much easier to not mess up if your arguments don’t get parsed by a shell before getting passed.

I'm being flippant, because shell scripts are so inherently error prone they're to be avoided for critical stuff like this.

If you _absolutely_ must use a shell script:

0. Use shellcheck, which will warn you about many of the below issues: https://www.shellcheck.net/

1. understand how quoting and word splitting work: https://mywiki.wooledge.org/Quotes

2. if piping files to other programs, using `-print0` or equivalent (or even better, if using something like find, its built in execution options): https://mywiki.wooledge.org/UsingFind

3. Beware the pitfalls (especially something like parsing `ls`): https://mywiki.wooledge.org/BashPitfalls

(warning: the community around that wiki can be pretty toxic, just keep that in mind when foraying into it.)

Try not to delete stuff with Bash.

This is the most reliable way. Bash has a few niceties for error handling, but if you are using them, you would probably fare better in another language.

If you do insist on Bash, quote everything, and use the "${var}" syntax instead of "$var". Also, make sure you handle every single possible error.

`set -e` will abort on any error, anywhere in the pipeline. It’s a must for any critical script.

Reply on news.ycombinator.com