Then there's flexibility. With bash you have a massive ecosystem of command line tools, including the ability to in-line code for tools like sed, awk and perl right there in your script. Python has a great standard library, but often the best way to do something in Python is using a third party module, so now you need to curate a module library across all your boxes. Also suppose your script needs to do things other than just process text, suppose I want to ssh onto a box and run a command? Is Paramiko installed, no? So now I need to invoke ssh in a sub-shell which is a PITA in Python compared to bash. Yes it's easier in later 3.x versions but now we're back to version soup again.
We have bash scripts running in our environment that are probably 20 years old, and if I write a new one now it will still work fine in 20 years time. Often it will be a quarter of the length of an equivalent Python script and much easier to understand and reason about.
Then again, there are many cases where a Python script makes a huge amount more sense, often run as part of a bash script.
For now I'm teaching myself CL tools, and I feel if I need to pipe more than 5 times in one liners I'd rather use Python.
Once things get complicated though you should probably push whatever transformations you need to do to a database.
The other day I set out to use BASH to mass rename, and I had such trouble backslash-escaping the dot characters and whitespaces to prevent sed from interpreting them specially. Eventually I gave up and searched for the pythonic way to rename them, and it was as simple as a string replacement followed by a call to os.rename() inside a for-loop. It was a breath of fresh air to escape “command line Kung Fu” and fearing the thrashing from shell globbing.
To be fair, I got my start in using sed and awk as powertools in the BASH command pipeline, but I don’t miss them compared to a language with strong data types and simple built-in methods for handling complex manipulation. Python is built in to basically every Linux distro that has BASH, and for the sake of simple transformations, it offers a lot of succinct methods that work on either 2.7 or 3.x with no external packages.
But if you're dealing with changing stuff in text files greater than 25 gigs I'd always recommend using the unix cmd line tool set for simple modifications if performance is a key concern. It works the same everywhere and can rip through big files. But I'm a Vim guy and I live in the shell so I'm surely quite biased lol.
There are plenty of pretty easy one liners though that can rename files depending on if you can define a pattern correctly though.
echo 'TV Show Name 2014.L337.H4XXX1080p-420.mkv' | awk -F . -v q="'" '{print q$0q" "q$1".mkv"q }' | xargs mv
find . -type f -name '*.mkv' -printf '%f' | awk -F . -v q="'" '{print q$0q" "q$1".mkv"q}' | xargs mv
python -c 'import sys; f=open(sys.argv[1]);print(len(f.readlines()))' .zshrc
But I wouldn't recommend python as a good one-liner language. So the question remains, what is awk particularly good at?
awk -F, '{ print $2 }'Now remember, this was popular when perl wasn't even something you could depend upon on all systems.
These days there's no reason you should even do anything terribly complicated in a nasty oneliner. Just save a nice function somewhere and call it with a concise alias.
Processing regular (i.e. machine-, not human-generated) tabular text files line by line. If you already know Python there are probably few reasons to learn awk.
One of those could be working in constrained environments where you have awk but no Python. Awk is part of POSIX and part of Busybox, so there are almost no environments which have a working Python installed but no Awk, while the reverse is common enough (e.g. embedded systems). Apple plans to remove Python, Perl and Ruby from the default MacOS install in the next version, but probably not Awk (POSIX).
(And I hadn't realized Apple was planning on getting rid of Python, Perl, and Ruby from default installs! Wow.)
For these jobs even an "import re" in Python is too much typing, compared to ubiquitous regular expressions in sed or awk. Awk is great at handling lines of delimited fields. Again in Python you would be writing your own boilerplate to read lines from files, splitting these lines, converting strings to numbers etc.