The growth of command-line options, 1979-Present
danluu.com
danluu.com
GUIs provide discoverability so often times you don't need a manual. If you don't know how to use a command line tool you must open the manual. So in that way I would consider the man pages as part of the UI and those definitely get cluttered.
For a desktop paradigm, there is always a way to do better with GUI as you add more features; but you'll refactor / redesign a lot on the way and that has a cost in familiarity, which directly impacts user productivity. It's a trade off.
The dirty secret of discoverability is that only a tiny subset of features is relevant at any time in the UX, and identifying that subset + triggering user awareness about it is the "art" part. We learned a lot with mobile in that regard because the harsh limit on real estate becomes incentive to identify the essentials.
Discoverability doesn't have to suffer from complexity of the software, but it's not just UI design, it's UX.
For some reason, designers and some users don't like it, but a hierarchy (folders, menus, etc) is something many people find useful and effective.
But it seems pretty common to declare people can only handle a few options plus search.
There are great domain apps (e.g. medical, games, professional audio, amateur astronomy to name a few I've seen) that really break open the format:
- feels like a tricorder from Star Trek, where small is good because it doesn't try to be bigger than it is, rather embraces its limitations are features not bugs (and to his credit, Jobs + Ive successfully translated these ideas into the iPod and later on iPhone early iterations).
- feels like a great videogame UI where despite the complexity, once internalized it's a breeze (even a joy, physically) to manipulate. It takes a lot of testing and feedback and is admittedly one huge QA cost in video game making, so there's why most companies don't do it for 'normal' products whose UI/UX is 'good enough' (or so they think, for now).
- clever integration with features, e.g.: slider on a phone with a tiny motor feedback when you cross a unit bar or even for magnitude; a bigger-deeper 'womp' around the center coupled with a UI magnet that makes you 'feel' and 'see' the slider fix itself on the 0 position. This all designed so you interact while looking at something else, just like a physical button.
I don't know. I feel like we've made steps of giants in the 2000s and then progressively dropped the ball when we realized the simplest UX on mobile yielded the highest engagement (think twitter, insta, how "simple" they are, how limitations actually enhance user activity; in a very questionable way but that's besides the point in terms of profit thus research).
I sincerely hope someday a disruptor kills everyone else with awesome UX (just like Apple did in the 2000s) and forces the market to re-think its abysmal UX standards, and let's not talk about ads because that would be extra-snarky.
If I were snarky I'd tell you that. But I'm not, so I'll leave these remarks for someone else to make.
The market is allowed to exist because profit is a quite decent proxy for utility. But whenever you are optimizing using an imperfect proxy, it's possible to overoptimize - make the proxy metric look better, at the expense of the actual goal it was supposed to stand for. I believe software (along with much of other market sectors) have crossed that point a while ago.
The economy to me is a vast "optimization space", so I can only agree. It's the 'trap' of a local minima I suppose, wherein the friction of moving away to find another possible (but uncertain) minima (hopefully overall lower or more acceptable ethically) is too high. So you've indeed overoptimized your way into a local pit (e.g. the ad model now taking over the news like a virus, resulting in this new "hybrid" object sometimes called "informercial" or "infotainment").
It seems that only "disruption" (of a magnitude worthy of the name) gives enough momentum to escape the steepness of a local historical/economical minima. Unless you've got some new S-curve (which quite literally may represent the math surface of this "escape path"), history tells us we remain stuck and move in circles about the center of 'gravity' of this local minima.
Dunno if that makes sense to you, pardon the loose physical analogies.
ZBrush feels like that to me. Most people think that's it looks like a dogs breakfast at first view, and that it's desperately in need of an overhaul.
Yet when you just want to sculpt 3D clay, it feels remarkably intuitive once you know where everything is.
You mean like Apple did in the 1980s? They regressed in the 2000s.
The UI is one input and the "computer" figures out the rest.
Regular users don't want/understand hierarchies, they get easily confused. They also don't want to organize things themselves, see the many failed attempts at tagging files, websites, images, etc. manually.
I am probably not a "regular user", but I got a job about a year ago defined by and intended for regular users, and one of the major parts of it was managing emails in a departmental account, using (you guessed it) hierarchical folders, and another was managing project documents in SharePoint, using (what appear to be) hierarchical folders.
I'm aware that people will tell you not to do the latter in SharePoint, that it's not really designed to be used like a filesystem, just use lists, etc. But purely from an anthropological viewpoint, I see how people shove everything on a shared drive, and then they try to move it to something like SharePoint and recreate a folder structure.
So, you know, it's not about what I want, but it's an objective fact people do want hierarchies, even when other people don't want them to. Google is pervasive, but it's not all of computing.
But the same could be said of CLIs tbh. After a couple dozen options, man pages, or `--help` lists become a dense read, where as a simple form on a webpage might be easier to grok. YMMV
A GUI can much more easily emphasize the most common options, it can visually group related options, and it can use common GUI widgets to, for example, show that some options are mutually exclusive with each other.
You don't read a man page top to bottom, you skim and skip around, discovering the information you need.
The trend in the command line is in an orthogonal direction - more and more complex function-like options with minimal mnemonic labels, optimised for text representations and text processing.
Which is a problem of its own, in a way. Reading a man page top to bottom is few minutes. Reading a user guide/Info pages (for the good software that still comes with one) for a more complex tool (e.g. gdb) takes couple of hours. If you're going to use the application more frequently or for more than an hour in total, the time spent reading the manual will pay for itself - in reduced frustration, much quicker discovery, and much less StackOverflow searches (and getting confused by wrong/misguided answers).
Reading manuals top to bottom: an ancient, forgotten superpower.
Of course some principles of speech and syntax should stay around (as they probably exist for a good reason), but it is foolhardy to grip tightly onto any language, human or computer, thinking it will always stay the same, when it plainly won't.
Partly, but less so.
Human language vocabularies approximate Huffman Coding - the more frequently used words (and phrases) tend to be proportionally shorter. This is a gradual process for spoken languages, and somewhat quantized for written languages.[1] It's possible because with human languages, "close" is "good enough"; one can read english written with ye olde orthography, if with a bit of extra effort.
Less so in the lifespans of particular programming languages. Per Postel's law we tend to be liberal with extending but conservative with breaking backward compatibility. I think the case of command line tool parameters is particularly conservative for various reasons; among them because the 'knowledge' of options is spread among multitude of separate project - many of them obscure and bespoke - and coordination would be nigh impossible.
However there are some signs of similar trends; note how the "function" and "integer" and similar keywords of older languages tend to turn into "fn" and "int" in newer languages.
--
[1] Not sure if there's a comparable process for grammars, but I guess it's a fair bet...?
Pejoration happens too. When is the last time you used a GOTO in code? And yet, 40 years ago, you couldn't write a BASIC program without one. And the revulsion against NULL causing a shift to Maybes and optional types to the point where the concept is reified in many languages now?
All I'm saying is that there are multiple mechanisms that are changing the languages we write code with, and that it is changing far faster than people realize.
So why does `ls` have a recursion flag? Why does `cp` have a recursion flag? Why does `chmod` have a recursion flag? Why does `chown` have a recursion flag? Why isn’t recursion its own command, that takes as argument the command to apply at each level?
Ditto untyped, untagged streams, which mean every tool implements its own custom UI behavior when attached to a terminal, instead of outputting tagged data and letting the consumer run it through system-standard data formatters.
This is not good—or even borderline competent—systems design. This is bullshit. Because the people who slapped it all together are bums, and the people who accepted it didn’t care enough to call them on it.
But noooo, we can’t change anything now; because “backwards compatibility”.
Protip: when you build a system that can’t evolve, that grinds to a halt under its own impossible weight, it’s time to confess “yeah, we totally ballsed that one up” and start over again, lessons learnt. Because maintaining that status quo is a tyranny: dragging everyone else down to the level of its incompetence, and beating them to death with intransigence. No prizes for guessing who that benefits: the bums who built it. And who pays for it: everyone else.
It’s a scam. Perpetrated by and for those who cannot admit they are wrong.
Smash it down. Because whatever comes after it should be so much better, the only question will be “Why the frell did we wait so long?”
--
“Those who make peaceful revolution impossible will make violent revolution inevitable.”—John F Kennedy
Why did ls even need 11 parameters to begin with? Why not exactly 1? Or none? Why did tar need 12? Why did find need 14? The 28 tools in the example (excluding non-existing ones) had an average of ~3 parameters per tool in '79. Should 2017 have brought us 200 extra individual tools to provide the same additional functionality? You want another tool that does recursion for ls to save us from "incompetent bums" and "grinding to a halt"?
Everything around you grew in complexity to adapt to your needs because there's value in consolidation. It's why you are now typing from a device that is orders of magnitude more complex and integrated than the one used to write down the Unix philosophy.
You ask the question "why" but fail to answer the "why not". Why would a tool that still does exactly what it used to do in the past not be able to also do more especially when it's still within its sphere of relevance? And not the "out of principle" answer, not the handwaivy, tin foil hat answer, not the offensive and insulting answer that you provided above (bums? incompetent? bullshit? scam?).
Times change and philosophies adapt to remain relevant. "We must use time as a tool, not as a couch" - JFK.
Yes I absolutely do. Remember UNIX? "Many tools, each small, each doing one thing well". Adding hundreds of flags to older tools does NOT achieve that. Your point here?
> Everything around you grew in complexity to adapt to your needs because there's value in consolidation.
Please don't cross into abstract philosophy. It adds absolutely nothing to a specific technical discussion.
> It's why you are now typing from a device that is orders of magnitude more complex and integrated than the one used to write down the Unix philosophy.
I use my computers as the complex messes and a series of train-wrecks that they are, and now hear me out please, because I literally have no other choice. Be it Win10, Linux, Mac, it's honestly almost irrelevant. One example: to this day nobody can make a reliable system-wide sleep mode that is not messed up by extra hardware (like my external HDDs), on neither of those 3 major OSes. In 2020.
I can't go back to my Apple IIc. I will lose my job if I do.
What's your point here? That such a complexity is inevitable? Is that it? But who argues against that? Of course it's inevitable, we see it in every area of life -- it's one of the aspects of entropy. What's being discussed is: "isn't it time to take a firmer stance and start using newer tools that don't have these problems and protest against the old ones by not using them"?
I personally answered "yes" and I am gradually doing more and more of it -- already replaced grep, ls, find/xargs, about 13 in total IIRC. You know what? My productivity on the terminal skyrocketed. I started ditching languages with too much magic and implied behaviour and too weak/random typing (PHP/JS come to mind) and moved to others which, while not covering everything that I'd like to have (like Elixir), are still letting me sleep that much better knowing that `[] == 0` is actually false and I won't have to curse my way through logs in the next morning.
> You ask the question "why" but fail to answer the "why not".
If you actually frequent HN and aren't just looking for conflict then you'd know that literally every month we have a few rants by people who are now burned out that decry the current state of affairs. But if you really want me to answer then I can likely summarise it to:
"Because our tiny brains don't work well with so much complexity in the long run. We can cope for a few years, some even for 20 and more, but eventually we can't. Because having smaller, more-focused, and more (total count of) tools that each do something really well, has been demonstrated to work better everywhere, not only in IT."
(But I am aware this has been contended endlessly and it devolves into belief and tribalism and science and historical evidence goes out the window in the first three comments tops. Sad. Shouldn't happen to techies. We should be better than that.)
> Why would a tool that still does exactly what it used to do in the past not be able to also do more especially when it's still within its sphere of relevance?
Because it's NOT, and your parent commenter gave you an excellent example which you immediately dismissed and THEN proceeded to ask your question again, pretending he never gave an example. Are you aiming at being disrespectful on purpose? Confusing.
> And not the "out of principle" answer, not the handwaivy, tin foil hat answer
You are aware you are crossing into ad hominem and almost insults, no?
> Times change and philosophies adapt to remain relevant.
Well that's a perfect example how the same sentence is viewed in two completely different ways. This is EXACTLY the problem, dude: philosophies don't adapt at all in our area, most of the time. I knew a sysadmin that adopted those newer CLI tools (most written in Rust but some in Go, like `fzf`) and he is now doing the job of 3 people -- especially after also adopting Nix/NixOS. And I know people that will defend their 5000+ lines of Makefiles with hacks around niche return values of `ls` with their dying breath.
That's exactly the problem. Most people (and philosophies) don't adapt at all. They prefer to fight very hard to mold reality to keep agreeing with their philosophy. I am surprised you are not seeing this because it's literally everywhere -- from your local airport with their laughable security checks to your local cafeteria that likely still uses cash registers that can be hacked with a simple cable and a laptop and running a script and the whole thing will take about 7 seconds before the lady behind the counter even gets back from the toilet.
> "We must use time as a tool, not as a couch" - JFK.
Again, your point?
---
Don't take my comment as an attack. But you very quickly dismissed some pretty good points and proceeded to [attempt to] hijack the discussion to irrelevant -- or too abstract -- examples. Again, I'd change my computer and OS to something more conservative, with one way of doing things and not hanging on to ancient and obviously not-working-that-well-anymore philosophies, tomorrow -- if it existed. But it doesn't.
The fact that now we have no choice doesn't prove your point at all.
Simplicity means different things to different people and at different times. You are both making the same argument for simplicity for the sake of simplicity with no objective measure for it other than a "guideline" philosophy of "does one thing well", and just dismiss the majority who doesn't fit your "burnout as proof" argument.
If you want to wave the philosophy flag you must also acknowledge that either it never applied to pretty much anything as such, or it implies you should adapt your tools to the times, the facts, the challenges.
> Again, your point?
Take full advantage of the times you live in or be left behind, burnt out, reminiscing the glorious past I guess...
Unix provides rather limited primitives for composition, and a lot of things remain unsafe or very difficult to do right in shell. That's why we keep extending tools and doing things (rather redundantly) in a real (if outdated and limited..) programming language.
A lot of that complexity is due to poor design, not due to inherent complexity in the user's needs.
Just yesterday in fact, while implementing a state machine. Except the developers of that particular language explicitly did away with GOTO, so I ended up approximating it with a SWITCH/CASE. At least the language supports fall-through...
Which is to say, people differ in judgements of various features (footguns vs swiss army knifes). Moreover, the judgements seem to oscillate as fashions, in ~10 year long cycles. Please recall how exceptions and templates used to be a hot thing back in the day. Ditto garbage collectors. Presently we're all in love with Rust, and the "JITs considered harmful" paper will drop any day now.
It's interesting to think about how things can be swapped dynamically in this system. Consider the evolution of 'telephone.' In the 1980's, 'phone' referred to a land-line telephone. In the 2000's, it was ambiguous; for young people, it meant 'cell phone' most likely, or we started calling them 'house phones' for land lines. Now 'phone' is mostly used for a 'smart cellular telephone'.
Another interesting example is "Playstation". That always refers to the current generation, and we retroactively apply "PS1."
It will be an interesting period for history in the future to try and decode our present-day meanings since so many words are period specific.
You could probably trace a similar evolution across programming languages, just by systematizing the phrases/words used in each to accomplish the same thing.
The true winner is probably gcc though, which was unfairly excluded from competition. Over 1000 options is excellent work.
https://unix.stackexchange.com/questions/112125/is-there-a-r...
As an exercise, try to sort a directory by size or date, and pass the result to xargs, while supporting any valid filename. I eventually just gave up and made my script ignore any filenames containing \n.
The `printf (od -> sed)' construction converts back out of null-separated characters into newline-separated, though feel free to replace that with anything accepting null-separated input. Granted, `sort --zero-terminated' is a GNU extension and kinda cheating, but it's even available on macOS so it's probably fine.
printf '%b' $(
find . -maxdepth 1 -exec sh -c '
printf '\''%s %s\0'\'' "$(stat -f '\''%m'\'' "$1")" "$1"
' sh {} \; | \
sort --zero-terminated | \
od -v -b | \
sed 's/^[^ ]*//
s/ *$//
s/ */ \\/g
s/\\000/\\012/g')
This is all POSIX as far as I know, except for the sort.
find /etc/ -mindepth 1 -maxdepth 1 -type f \
-printf '%s:%T@:%p\000' \
| sort -z -t: -k1,1g \
| sed -E -z 's/^[0-9]+:[0-9.]+://g' \
| tr '\000' '\n'
And really the answer given at the stackexchange link is quite on point:
> However ls is really a tool for direct consumption by a human, and in that case further processing is less useful. For futher processing, find(1) is more suited.
Find is the right solution, find gives you an enormous amount of control, to re-implement find in ls seems counter productive.
python -c 'import os; \
os.write(1, b"\0".join( \
sorted(os.listdir(b"."), key=lambda p: os.stat(p).st_size)))' \
| xargs -0 ...
(Python3 does try to make this hard on us — Unix path names are just bags of bytes, but Python insists that printing must have some known encoding. I get around that by abusing os.write(1, ...) and by adding the dummy b"." argument to os.listdir(), which forces the results to be in bytes format.)
I think we can all do that. Did anyone ever came across a legitimate reason for filenames with newlines? All I can think of are intentionally bad-named files to exploit bugs. Those filenames should be avoided in the first place.
What if the largest file on your drive has that filename and no tool can show it?
Another example: the modern Bash shell vs the Plan 9's Rc shell. The difference in documentation is over tenfold, with the latter still being more less laden with gotchas:
$ man bash | wc -w
48189
$ 9 man rc | wc -w
3264
That's an advantage; Plan 9 provides awk for those use cases. There is a clear separation, and a simple yet powerful language for communication (pipes, file descriptors, and files).
>bash is trying to be a full scripting language these days
Arguably Bash over-did somewhere around the time they added support for TCP connections
exec {FD}<>/dev/tcp/news.ycombinator.com/80; echo GET / >&$FD; cat <&$FDI won't get to this in Oil (without help) but I still think it's valid. The languages have so much overlap that they should just be combined. Of course that's what Perl was (leaving out Make), but Raku and Python 3 seem to have left a hole in that design space.
Shell, Awk, and Make Should Be Combined http://www.oilshell.org/blog/2016/11/13.html
Example Code in Shell, Awk, and Make http://www.oilshell.org/blog/2016/11/14.html
$ man dash | wc -w
10122
$ ls -lAh `which bash`
-rwxr-xr-x 1 root root 1.1M Jun 6 2019 /bin/bash
$ ls -lAh `which dash`
-rwxr-xr-x 1 root root 119K Jan 25 2018 /bin/dashperl -lane # process input by line, split into fields, print outputs with new lines, execute code
netstat -planet # show process, extended info, all socket types
ss -pimento # basically every piece of info about live connections
iostat -txyz # extended, omit first report (the since-boot one), omit devices with no activity, log datetime
ps aux
rsync -rave ssh SOURCE DESTINATION less -SEXIER
(The second E does nothing but complete the word)
I think the change was jokingly called “pulling up the ladder behind us.”
Learning some command line commands feels like learning a whole new framework. A man page should rarely need to be a whole damned manual.
If so, then this primarily seems like an issue of how to present the documentation and not the mechanics of the change itself. Instead of autogenerating docs that say concat :: Foldable t => t (t a) -> t a (or whatever) and making people think about Foldable, write docs that say that concat is still [[a]] -> [a], with a note at the bottom saying, by the way, this generalization exists, click here to see the more complicated version if you've decided you care about Foldable. Probably the text of the docs remains the same, in fact ("concatenate a list of lists together into a single list"), you just display the real prototype and explain that "list" could also mean something else. (There's a number of ways to implement this, from writing docs by hand, to extending the doc protocol to support this specific case of lists and Foldable, to extending the doc protocol to support documenting a fake function prototype and then have a link to the real function at the bottom, and using the type system to ensure that the fake prototype is just a special case of the real function.)
A corollary is that documentation presentation is an inseparable part of library / tool design.
The same type of solution that you propose for docs can work here too, though. The compiler can trivially substitute the typeclass for any known instance.
tar xvf foo.tar.gz # extract, verbose, file
tar -x -z < foo.tar.gz
tar --verbose -x --xz < foo.tar.xz
This makes it easier to remember the syntax for create and list too:
$ tar --create e.txt k.txt > out.tar
$ tar --list < out.tar
e.txt
k.txt
https://www.oilshell.org/release/0.8.pre2/doc/INSTALL.html
-----
So basically tar can behave normally (at least GNU tar can) -- it's just that there's an ancient idiom that most people have in their fingers, and that's copied around in various instructions.
The nice thing about the verbose way is that the hyphen prefix enables autocomplete.
tar caf foo.tar.gz foo/ bar/ # Compress As Fuck
tar xaf foo.tar.gz # Xtract As FuckThe alternative (presuming the extra functionality is useful to someone, somewhere) would seem to be more standard commands, and that choice would use up more of the precious global namespace for small yet memorable lower-case command names.
I prefer as much of that as possible be reserved for userspace. I'll admit my mind boggles a bit when I ask myself what tar could possibly do with 139 flags, and it makes me a bit nervous about the surface area the program exposes.
To better expand on what fctorial is probably trying to say, argv is a simple list of strings. Plain text is a nice common ground among the myriad of programming languages that executables can be written in. In order replace options with "a structured, typed, reflective API", argv could no longer be a simple list of strings. There would have to be a common typesystem among all programming languages. It wouldn't just be needed for the understanding of argv, but also for the output, since the output of programs is used to feed the arguments of other programs. This puts heavy restrictions on the liberty programming languages can take in defining their own typesystems while retaining compatibility with such an inter-program typesystem.
This isn't to say that it can't be done, but I'm fairly certain it would suck on multiple fronts. You might be able to gain some things, but you'd certainly be giving away others.
EDIT: I don't want to spend too much time expanding what you'd be giving away, but to start
1) Having the common ground be plain text means that you get 1 form of output that can be used both for human and for machines to read. The programmer might not even intend for their output to be used by other programs, but it will always be trivial to do so. You can propose that they could have 2 forms of outputs, one plain text for humans and one typed for machines, but there would be no guarantee that they would have the same information. It's likely that one form will miss information that the other has, and you'd be left with humans reading output meant for machines or machines reading output meant for humans, so you gain nothing.
2) Less brevity. You complained about options being too detailed ("remembering copious minutia"), but that's because they're meant for interactive use. This is not a problem about the interface being untyped. It's about the interface optimizing for being quick to use. This would either also arise with a typed interface (although more difficultly so) or the CLI would simply be more tedious to use. No win there.
3) It would be more inconvenient to use interactively. For example, if you want to differenciate between a number 5 and a string "5", you're going to have to add syntax, like the quotes I used in this sentence. You can see the inconvenience by setting you shell to be something like python, for example. There would be too much syntax for interactive use.
So it does, albeit a better example of the shell doing something nontrivial would be replacing file patterns with filenames, but it doesn't help since my program could be expecting a list of urls instead and has no standardized way to tell the shell this.
>To better expand on what fctorial is probably trying to say, argv is a simple list of strings. Plain text is a nice common ground among the myriad of programming languages that executables can be written in. In order replace options with "a structured, typed, reflective API", argv could no longer be a simple list of strings. There would have to be a common typesystem among all programming languages. It wouldn't just be needed for the understanding of argv, but also for the output, since the output of programs is used to feed the arguments of other programs. This puts heavy restrictions on the liberty programming languages can take in defining their own typesystems while retaining compatibility with such an inter-program typesystem.
Yes, a language-agnostic reflective type system with a shell and supporting language tools, it's a pretty big implementation and coordination effort and an even greater one to retroactively describe the command line semantics of existing commands so not holding my breath here.
>Besides that, it would be more inconvenient to use interactively. For example, if you want to differentiate between a number 5 and a string "5", you're going to have to add syntax, like the quotes I used in this sentence. You can see the inconvenience by setting you shell to be something like python, for example. There would be too much syntax for interactive use.
A matter of preference I suppose as I have been using jupyter/ipython for all "shell" tasks for quite a while.
Well, if I don't use it often, I'm not going to be looking at the documentation often. However, when I do, we're talking about the time it takes to type `man foo<enter>/bar<enter>`, where bar can be `keyword` or `--option\b`. That doesn't even take 30 secs.
Does this better alternative exist?
>the added options are convenience flags: setting the permissions of the new directory and making parent directories if they don't exist.
Obviously completely unnecessary for normal users, but those have to exist for automated processes. The "problem" is caused in part by the success of Unix, that now computers are using computers.
function mkcd() {
mkdir -p -- "$1" && cd -P -- "$1"
}Sure it’s convenient for writing scripts too but a script could just as well include some boilerplate for the functionality if required and it if not.
This is the argument between CISC and RISC.
> To be clear, I'm not saying that I or anyone else could have done better with the knowledge available in the 70s in terms of making a system that was practically useful at the time that would be elegant today. It's easy to look back and find issues with the benefit of hindsight. What I disagree with are comments from Unix mavens speaking today, with comments like McIlroy's, which imply that we just forgot or don't understand the value of simplicity, or Ken Thompson saying that C is as safe a language as any and if we don't want bugs we should just write bug-free code. These kinds of comments imply that there's not much to learn from hindsight, we were building things as well as anyone could build things in the 70s, all we have to do is go back to building systems like the original Unix mavens and all will be well. I respectfully disagree.
In other words, the real problem isn't that modern software is getting more complex (as there is a good reason for it); the problem is that we're insisting on using an environment that is not up to the task of dealing with modern complexity.
For an example of how insane this world is: consider putting your source trees in a location that contains a space in its name. How many build systems are going to completely and utterly fail to build your code? And... why should we tolerate this situation?
anything as powerful as FFMPEG, EXIFTool, ImageMagick, x264/265, etc is going to have switches/options from hell just out of plain necessity. these are not simple little Unix-esque tools. these are full blown applications that just happen to not have a UI.
It's not as much that the arguments allow you to run a program under variant configurations, but rather that you create completely new programs for each run.
The alternative is to hide all of the complexity and hope that the defaults are good enough for your users. Which works ok for stuff like Mom encoding her cell phone video but falls down pretty quick when someone needs to do something slightly out of the box, like add in a second audio track.
$ import fs
But if you do this:
$ import fs as myfs
$ myfs.vim
A general purpose ad-hoc data format with all the fundamental types, that doesn't require endless text parsing but can be converted to a 1:1 text format for human viewing, would be helpful in so many situations, including serialization for piping between unix tools.
I was a bit of a wuss and had to use x.409 dumps, my boss could look at the raw data packets and debug problems.
Although not technically a pro/con list, it highlights the evaluation criteria used to compare.
[1] https://github.com/kstenerud/concise-encoding#comparison-to-...
[2] https://github.com/kstenerud/concise-encoding/blob/master/de...
In your comparison you claim Concise is zero-copy, but it does not appear to be so, at least in the sense that Cap'n Proto and FlatBuffers are. Concise appears to use variable-width integers, tag-value sequences, and 8-byte alignment, all of which make encoded messages unsuitable as in-memory data structures; it looks to me like you have to parse the message into some sort of AST before you could meaningfully operate on the content. "Zero-copy" in the Cap'n Proto / FlatBuffers sense means that there's no need to do any "parsing" because the whole message is efficiently traverseable as-is; e.g. you can mmap() in a very large message and then find and access any one value within it in O(1) time (or perhaps O(log n), if the data structure is nested). This generally requires that primitive values have fixed widths and fields have fixed offsets within their parent object.
Digging into your docs it looks like what you really mean by "zero copy" is that individual strings or byte sequences can be used directly without copying them out of the original message buffer. This is a fairly common property of any binary protocol; e.g. Protocol Buffers can do this (albeit without NUL-termination), but in your table you have indicated that it cannot. I would maybe call this "zero-copy strings" to be clearer.
On another note, in your table you have indicated that Cap'n Proto doesn't have a "String" type, but this is not true -- Cap'n Proto's "Text" type is a UTF-8 string (and is even NUL-terminated on the wire).
(I'm the author of Cap'n Proto.)
XML strength also lies in its extensibility, and its large number of compliant implementations, which goes way way beyond a single uncompleted implementation by a single author.
RON is fine for rust, but not for other languages that might not have its feature set.
I personally don’t use most of these options and their existence just clutter up the manual.
https://www.gnu.org/software/coreutils/rejected_requests.htm...
tar xfvz my-source-code.1.2.3.tar.gz
The reason is arguably that tar has too many responsibilities here. Tar should only do two things: create tar streams and extract tar streams into files. You do that with "tar c" and "tar x", and that's everything you should need to remember (ok, fine, "tar t" for listing files is good to know as well).
Commands to create tarballs should look like this:
tar c directory | gzip > directory.tar.gz
zcat directory.tar.gz | tar x
cat directory.tar.gz | gunzip | tar x
One of the benefits of the "swapping out" parts are of course that you're not restricted to what's built into tar: you can use ANY compression utility you want. For instance, there's a thing called pigz[1] which parallelizes gzip decompression for increased speed. It's trivial to swap into the pipeline when you use tar that way:
cat directory.tar.gz | unpigz | tar x
We've taught a generation of programmers that "tar" is hard to use and you have to look up a magic incantation every time you need it. Ironically, that makes tar less flexible, because now you think of it as a single-purpose utility (creating and unpacking compressed tarballs) instead of a versatile unix tool which can be used in all sorts of different situations.
Modern tar can detect the format so "tar xfv" is enough. The "v" option only means "verbose" so it's better just "tar xf" and avoid spamming the terminal.
Note: If none of your files were clobbered and there are no spaces in the filenames to worry about you can do:
tar -tf [tarball] | xargs rm -d
Run it repeatedly if you also want to remove the directories.
But what I came here to say is that I find the typical tar flags pretty intuitive. X means extract, z means using zip/unzip, v means verbose, and f for file I/O instead of stdin/stdout. For typical use cases, I don't find its interface magical but logical. I'll freely admit to needing 'man tar' anytime I stray from the typical use, but that's true of a lot of commands.
The bigger offender in my mind is rsync. I remember auvP as my typical flags but don't ask me what they do. And good luck guessing whether my destination or source path need a trailing slash on the first try...
I remember reading that xkcd comic though and I went "yup, that's me!", and I've spent a bunch of time thinking about why that was. You list those flags there and say there intuitive, but they're unique to tar, you have to memorize them, and there's no reason that they need to be in tar when they're available right there in the shell.
Why do I need to memorize that the "f" flag is "file output/input", when I can just write "> dest-file" or "cat src-file |" (or "< src-file") in my shell? That works for every command, not just tar. Why do I need to remember that the z file is for gzip (what's the equivalent flag for bzip2?) when gzip/gunzip exists and works for anything?
So many people simply don't understand what tar does or how to use it without googling the magic formula they need, and it's because of all these damn flags that everyone insists on using instead of doing it the "unix way".
dd is another program where the commandline flags predate modern usage (although it does at least behave nicely with pipes by default) and look totally weird compared to everything else. But again nobody wants to touch them because it would break millions of existing scripts.
> z means using zip/unzip
gzip/gunzip, doesn't look like tar supports zip/unzip, which is a whole different algorithm.
Perhaps you mean that you've looked it up. Others could do so, too; it's not that hard (even if a web comic makes light-hearted fun of the subject)!
Anyway, here's a fun fact: tar is smarter than you give it credit. The commands below do just work:
tar xf uncompressed.tar
tar xf gzipped.tar.gz
tar xf xzipped.tar.xzHow about:
tar c folder | bzip2 > folder.tar.bz2
tar -c [files...] -f archive.tar
tar xf archive.tar.gz # to uncompress
tar caf archive.tar.ext [files...] # to compress
Back before clear, I used 'echo ^V<esc>c', but that's not portable to different termcap/curses/ncurses display devices.
that said, I live in it.
Given the nature of open source, I'd be surprised if there isn't a alternative for less dynamic, less resource req, systems. Is there a system out there that still uses fewer commands?
I don't know how to ask this next question, so please bare with me.. Are there any modern systems that don't require modern hardware, but are just more efficient versions of past systems?
I have been happy with the way I could use my computer for the last 20 years (probably earlier, too).. so are there any efforts to keep the simplicity of older systems that were adequate for most, make them more efficient, and either have "super hardware", or I suppose very cheap hardware?
Thanks for anyone who made it through my ignorant ramblings.
Can't find but feel there's a quote like "Programs will grow to fill memory"
BTW, I have a first gen iPad sitting around that worked very the first two years or so of its existence, and it barely functions today. I know part of that is Apple "protecting" older batteries by slowing down older stuff so we go out and buy new things, but I would like to believe that we're capable or creating an OS for that hardware today that would run better than it did originally, rather than barely function at all.
That is a ball we set in motion when we introduced virtual memory, particularly with overcommit.
Give a developer the option to write calls to malloc that never fail, and boy will they make use of that! Performance be damned.
I remember when I ran the original MacPaint (I assume it was in hand optimized assembler?) on a 16 MHz 68030 and it was amazing - everything seemed to simply be instantaneous. It put in perspective how when the 68000 came out, it was a "minicomputer on a chip" and the 68030 was conceived as a "mainframe on a chip".
Also tar's increases might include new compressions methods like bzip2, XZ, etc.
Most of the tape devices themselves had/have compression on them so that tar did not need to do it. However, that compression was pretty weak (2:1), so giving tar the ability to use bzip2 and send the compressed data stream to the tape (bypassing the hardware compression) gave the user much more convenience. Also, older systems probably were bad at doing the compression in real-time, so it probably slowed down the write to tape. I'm just guessing, I have no experience writing to tape from such an old system. I'm not really upset by tar geting some extra features.