I use shell because I have to, not because I like it. I dread maintaining shell scripts which have a bunch of awk and sed in them.
The Unix ideal of small single-purpose tools and text processing is separable from these old warhorses.
I use shell because I have to, not because I like it. I dread maintaining shell scripts which have a bunch of awk and sed in them.
The Unix ideal of small single-purpose tools and text processing is separable from these old warhorses.
So for example, selecting columns is easy, and can be done by name.
Here's a useful little snippet demonstrating converting and processing the CSV output of the legacy "whoami" Windows command. It lists the groups a user is a member of without poking domain controllers using LDAP. It always works, including cross-forest scenarios.
WHOAMI /GROUPS /FO CSV `
| ConvertFrom-Csv `
| Where-Object Attributes -NotLike '*used for deny only*' `
| Select-Object -ExpandProperty 'Group Name'
It looks like a lot of typing, but everything tab-completes and the end-result is human readable. I find that people that prefer terseness over verbose syntax are selfish. They simply don't care about the future maintainers of their scripts.PowerShell can also natively parse JSON and XML into object hierarchies and then select from them. That's difficult in UNIX. The Invoke-RestMethod command is particularly brilliant, because instead of a stream of bytes it returns an object that can be manipulated directly.
$p = Invoke-RestMethod 'https://ipapi.co/8.8.8.8/json/'
echo $p.org
echo $p.country_name
PowerShell Core is available on Linux and is great for ad-hoc tabular data processing! Give it a go...I'm a pretty advocate for how it works in these use cases now.
Generally I still use Bash\Sed\AWK for the one off instances when ever I need something done there and then. But if I'm writing something that's going to be involved in my CI\CD with a high chance of needing to be maintained. I'll nearly always write it in PowerShell.
Even better if I can write such utilities in a more modern programming language like Go or Rust, if the organization has personnel with expertise in such languages.
As an example, you have some data you need to fix as input to some program. you incrementally try to fix it with perl:
1) run program with data, observe errors, infer what needs fixing ->
2) write perl to fix data, modify data ->
3) repeat from (1) until no errors
You have no expectation you'll ever need to fix data corrupted in the same way ever again.
Also rectang: "Even better if I can write such utilities in a more modern programming language like Go or Rust..."
Thank you for demonstrating how Linux decays due to lack of experience and NIH syndrome.
For me, that's THE reason to use it. It's terseness is what allows it to be efficient enough to be used primarily interactively.
And anything more complex than that I typically just use python.
I never understood the whole religion around unit tests. Integration tests are often far easier to write and far more valuable.
Like you said, unit tests are really nice when testing for a known, expected output.
Unit tests that are effectively testing mocks and crazy stubs because your method has side effects? Not for me.
You need both unit tests and integration tests.
I have written a tool (also a script) myself that allows you to write unit tests, manage your scripts in git, load scripts in other scripts, etc. Maybe I will post a Show HN in the coming weeks, but at the moment I would like to round up some edges before posting it.
In my experience, the biggest problem is that there are many different runtime environments out there that differ in detail and make it hard to write scripts that run everywhere. But programmers not applying all their skills (e.g. writing tests) to build scripts are also part of the problem.