I've seen this so many times in self-rolled auditing kludges that it has become kind of a running joke. I'm now surprised if anyone does anything more sophisticated than sym encryption of master credentials used to guard secrets without using the same secret for both as a first pass. Embedded master passwords abound in most legacy OSS configurations.
Texas is a long, uncomfortable haul in general. The year I did this crossing was 1994. Witnessed racism, religion gone wrong and malicious violence in more than one local personality jumping on for a town to town hop.
New Orleans coach was 2004. The rowdiness, smells and movement in the train gave me the only migraine I've ever had. I could barely give directions to the cab driver for
my hotel and slept 16 hours.
Public ground/rail transport in the US is not for the
faint hearted. Spoken as a travel veteran in three decades
of cross country trips via Greyhound and Amtrak.
TIPS:
* Don't cross Texas in a Greyhound.
* Don't go to New Orleans Amtrak coach from DC.
A lot of people are overlooking the ultimate consumers of this data or being the usual SV shills. This data and all PHI/PII can be resold to insurance packaged in slightly different ways. Have fun with your 20th century issues in the 21st.
Actually 10% of the sysadmins make the other 90% look pretty good in this respect. I've woken many mornings at 4 am without an immediate rationale to realize a moment later that there could be a problem and it could be mine. The SA mind is the best arbiter of possibility within the open domain unrealized by data learnage.
This is pretty classic amateur hour. He basically says 'I now know docker and everything looks like a container! Look at how productive I can be by discarding this set piece environment with out of fashion, slow and deliberate provisioning for docker images that I can create once and run forever with auto provisioning and dutch oven magic + an editor!'
Still using LXC where it makes sense and never felt the call of Docker...or Kubernetes or any of this developer popularized opaque tooling over sophisticated meddling with my systems.
Self signed internal CA for private use all day + mandatory client certs. Internal approved CA for business. Never see a need to change this. This Lets encrypt stuff is faddish to me.
When I was born it was still possible to learn your trade (engineer, programmer) without too much onus if you didn't attend college. Without college now your options are strictly limited. This is one of the reasons why we have a real problem in the US: those who are capable and self/job taught but discriminated against and those who didn't have (self imposed or otherwise) difficulties during school years and were encouraged or raised for success.
I have't looked at it rcently but the last time I checked systemd was the 'nice to have' that was mandatory for some of the software I associate with a fully fuctional desktop which is not just a WM, DE (or not) and basic applications like xterm , vim, etc...But you are right. You can run the linux desktop like its 2007 without systemd on select distributions without screwing around too much...of course don't add new software without researching dependencies and default build options or you may end up with the worst of systemd dependencies anyway. At that point I just run KDE on
FreeBSD.
Doesn't sound like performance was your end goal unless it was to demonstrate that your understanding of the unix tools could produce a solution slower than a custom built solution in the language of champions.
I was going to be the long haired *nix geek here but I have no hair and the world is moving on. I can't pick bones with python for data science/analysis and personal convenience. _However_ as a principal engineer if someone was to say, for a trivial dataset, that we need python and pandas for an operation like this where python + pandas was not already provisioned the answer would be no.
I always want a nice high performance vehicle with luxury features as long as I can drive. Pub transport is not an option for anything but work and cars are one of the last pleasures in the gen X life.
In my world your case is the corner case.
Don't think I want to deal with multiple terabytes of data as it's provenance is questionable and this data services industries that the human population did without until about ten/fifteen years ago.
This is something I've noticed in the last 8-10 years. The rise of the python/js/java paradigm everywhere. Some of the associated LDIF (json) I enjoy much more than XML and flat files but the misapplication of tools is becoming an epidemic. When I can write: awk -F "," '{for (x = 1 ; x <= NF ; x++) {if ($x ~ /[0-9]+/) {a[x] = a[x] + $x}}} END { for (p in a) {printf "%d = %d\n",p,a[p]}}' to sum columns in 5 seconds and people are scrambling with libraries to do matrix operations I tend to scratch my head and walk away. The aversion to the command line is also something that bothers me but I don't run into it as much in my field.
A systemd conceptually is quite a good idea. However systemd is not that systemd. Much rather go with rc.conf + script approach, disable all systemd services, lock selinux against systemd socket activations and run a purpose controlled environment. Avoiding systemd does force one to not use Linux as a desktop..oh well.
At 50, I have noticeable fast and slow periods. But when everything is firing correctly the correlations and ability to quickly see the forest for the trees is better than at any other time in my life. People are also more interesting and I tend to enjoy each nice day a little more than the last. Most importantly for professional work I now know when to give up and move on or dig in and finish.
Good. We don't have to worry about node.js, apache whatsit and Google go-away v1.0 being our goto technologies in a critical space. That's progress. Stick a rest/rpc interface on the outputs and ctl and attach a wireless dongle to a middlebox: voila modernized! If you really want to get fancy attach ELK to the 'stack' and put what data you want in it to represent something or another. Mission accomplished!
Back in the 90s and early 2000s there was a good market for desktop apps based mainly on glue languages and an RDB + interface. Most of these were utter crap. From looking at this offering and other (P|S)aaS nothing has changed except the venue.
A little bit of know how goes a long way towards process improvement. When you have what process improvement looks
like forced down your throat, mostly contrary to common
sense, is when you know you work in enterprise software.