The Collapse of the Unix Philosophy
kukuruku.co
kukuruku.co
Back when UNIX was born the <tab> character was a first class citizen in every computer on the planet, and many languages used it as part of their syntax. Static binaries were invented when Sun and Berkeley co-developed shared libraries and there needed to be binaries that you knew would work before shared libraries were available (during boot before things were all mounted, during recovery, etc)
It always amazed me when someone looks at computer systems of the 70's through the lens of "today's" technology and then projects a failure of imagination on the part of those engineers back in the 70's. I pointed out to such a person that the font file for Courier (60 - 75K depending) was more than the entire system memory (32KW or 64KB) that you could boot 2.1BSD in.
Such sillyness.
Writing a mere string out to a file on a non-UNIX was nowhere near as easy as ‘fd = open (“a file”, O_WRONLY); write (fd, p_aString, strlen (p_aString)); close (fd);’ on UNIX. Many systems required either a block-oriented or a fixed-record (with the record structure to be defined first) to be opened, the block or the record to be written out and then the file to be closed. Your record-oriented file has grown very large? Brace yourself for a coffee break after you invoke the “file close” system call on it. Did you process get killed off or just died mid-way through? Well, your file might have been left open and would have to be forcefully closed by your system administrator, if you could find one. Your string was shorter than the block size, and now you want to append another string? Read the entire block in, locate the end of the string, append a new one and write the entire block back. Wash, rinse and repeat. Oh, quite a few systems out there wouldn’t allow one to open a file for reading and writing simultaneously.
Flawed make? Try to compile a 100 file project using JCL or DEC’s IND using a few lines of compilation instructions. Good luck if you want to have expandable variables, chances are there wouldn’t be any supported. You want to re-compile a kernel? Forget about it, you have to “generate it” from the vendor supplied object files after answering 500+ configuration related questions and then waiting for a day or two for a new “system image” (no, there were no kernels back then outside UNIX) to get linked.
Awkward UNIX shell? Compared to crimes a numbers of vendors out there committed, even JCL was the pinnacle of “CLI” design.
No matter how perfect or imperfect some things were in UNIX back then, hordes of people ended up running away screaming from their proprietary systems to flock to UNIX because suddenly they could exchange the source code with their friends and colleagues who could compile it and run within minutes, even if some changes were required. Oh, they could also exchange and run shell scripts someone else wrote etc. In the meantime, life on other planets was difficult.
That was well before I met AS/400 and MVS.
And then I had contact with a Burroughs A-series and it's appropriately named OS, MCP.
OTOH, I love the 3270s.
True enough, but as a younger programmer, I find it pretty reasonable to look back at computer systems of the 70s and wonder if we can do better today. I feel a little bit gross every time I have to write a bash shell script (or edit config files that aren't JSON/XML/YAML, for that matter), and I don't think that's a bad impulse. That something so inelegant and unsafe is still in widespread use in 2017 really ought to be a scandal. Even if the author didn't frame the issue in the most charitable way for the earlier trailblazing generations, he's calling attention to the right issues.
In other words, if you couldn't justify something being designed a certain way de novo, why be content with the existing design!?
TOML is my preferred config file language option where I have a choice - https://github.com/toml-lang/toml - but I suspect that suffers a lot of the same problems.
The modern solution would be that there needs to be a network configuration tool which generates specific configurations for each component, is capable of encoding arbitrary invariants, and works consistently. Configuration also needs to be "push" based on events - when a DNS server dies, it should be able to figure out "we need at least 2 DNS servers, we have 1, fire up a new one - then update all systems to know about the new one".
Configuration management systems for Linux, by and large, suck. They're very good at starting from an Ubuntu Server install and building on that, and then get more and more fragile as the system lives on. Some of them (Saltstack, for example) do have some degree of event management - you can run certain commands on certain things happening, but it's not declarative or reactive in the way you'd hope - e.g. you can't just say "this system knows about all DNS servers" and expect it to work. The Docker/Kubernetes ecosystems claim to solve the network configuration problem (in a really awkward roundabout way), but not really intra-system configuration, and it still takes a lot of manual work.
NixOS gets a lot closer - but it needs to be expanded with a constraints solver and an event processing system. It's Turing-complete, so you can encode pretty much whatever you want into it, while still being a reasonable configuration language (basic use of it looks a lot like JSON).
But the point is - the formats individual components use for configuration should be more-or-less irrelevant. They could be completely opaque, so long as it's possible to get from the network config to the individual component's config and it's possible to update that config on-the-fly. In fact, it'd be more useful to standardise on one library which can handle all that for you.
What about Lua tables?
As a relatively young programmer i look at the computer systems of the 70's and think to myself "what the fuck are programmers thinking when writing today's programs". I mean we got amazing tools and a plethora of amazing languages, an insane amount of memory and an idiotic amount of processing power, chips specialized for sound and even ones for visuals. And yet everything lags.
As for shell scripts, they are fine. Not that you have to use them, as you can use python, scheme, anything (even C with tcc).
As for XML... oh god.. it is the worst, the absolute worst, way to do config files. Jesus help, is it awful. Bad for the user, bad for the processor. Simple key–value pairs (as in "power_level: 9000") are good enough for almost all configuration. Hell even the INI format is faaaaaaaaaaaaaaaaaaaaaar better then XML (and YAML, JSON (although these are themselves better then XML)).
I look at the 70's and think "haven't we learned any fucking thing ?".
PS On topic, the Unix philosophy is still valid. For a plethora of reasons. I typed too much already but if anyone's interested in my "opinion" i'l write it later today. PPS The author of this piece doesn't even know what "the Unix philosophy" exactly is.
The lag should be regarded as intolerable, but instead it's routine. I shouldn't have to wait for computers like this.
Shell is a nice idea, but UNIX should either have banned whitespace in filenames or made the shell handle it properly. It's amazing how many things blow up on encountering a space, and you can upset people even further by putting newlines in your file names.
The difference is in the old days, you adapted to computer. Now, computer must adapt to you.
Take a look at MS's latest text-editor program's release notes (https://code.visualstudio.com/updates/v1_10). They have to build a program that accommodates so many different existing must-have features (from a potential users POV), and for each feature, accommodate almost every flavor of configuration that has ever existed for any other text editor whose style the potential user is now "locked-in to". Otherwise user will refuse to adopt.
In the 70's, I assume the audience was a pretty tight knit community of computer engineers, coming from a time of very frustrating tooling where it wouldn't be uncommon to lose a day's work because the 'Save' function failed or something like that. In this environment, you learned the crazy unintuitive keymap for vi and called it a blessing.
Now, you yell at your editor if it doesn't remind you to save your work when you close it. Or you yell at it when it does remind you; too many pop-ups! The point is that user preferences are more fragmented and expectations are higher for features, and willingness to learn details and gotchas of a program are lower, which has created the bloated program for the simple things like text editors.
Instead of writing bash scripts, learn awk.
Awk is old, much older than bash, and has largely gone unchanged since '89.
But, despite the fact it was born in an era where we were still nailing down PLT and what was important in a language, it ticks a lot of boxes.
Having written some sizeable (5000+ LOC) programs in it, it kind of feels like a dynamic mash up of C, and Lisp.
Try Awk, and see if quick scripts are still as painful.
Cough... systems... cough...
We can do better. Plan 9 existed. Apple's Newton has a unique take on data files. AS/400's map all storage to memory addresses giving a single address space for everything. With Smalltalk, applications would just be new classes you import into your system.
The key to Unix's success seems to be not doing better, but doing as little as possible while still remaining useful.
And yes, it's a bit disappointing that, when we did better, we were less successful. Today, all the dominant OSs are children of either Unix or VMS.
And MVS, which went nowhere, but in the sense it's always been there and, it seems, always will be.
You have to define "better". Is it synonymous with "easier", "type safe", "less resource intensive", "object oriented", "functional", "faster" or some combination thereof.
How will you maintain backwards compatibility? Because, despite the fact that you may have just graduated, there are scripts and programs that rely on scripts that go back decades and people are NOT going to rewrite them just because of your new whiz-bang solution.
You may use it and if you represent a large enough organization, it might be viable for that use case but if you're only shopping a solution in search of a problem, you may as well not waste your time.
Back to Plan 9, you can expect something that is just slightly better than your current options, and that keeps being this way to steadily gain adoption until it's popular. But that requires it being no worse than the current on any popular use case. Plan 9 may have a great architecture (I'm not sure about that anymore), but OSes compete on many different fronts, like portability, performance, software availability...
If the Plan 9 architecture were something amazingly better that solved a lot of problems, you could be sure some people would have adopted it, and then it would start improving on the other dimensions. But it's not. It's better. It will make your life easier. Yet it will bring no order of magnitude change. So, people simply take a deep breath and write (again and again) the 3 or 4 lines of code required to solve their problem in Unix, instead of changing everything.
Please, no. Don't convert system programming/engineering to the mess that is web development today.
Because backwards compatibility is more important. We can't just throw everything away every couple of years. Besides, who is to tell that the newer design will be better? Judging by the history of our industry, I have some serious doubts.
As much as it's popular to think otherwise, this isn't true. Taken as a whole, newer things are more often than not better than older things.
For example, cryptography has been on a huge march upward since the days of Unix crypt().
Browser design is a mess, but that's because it's a battlefield between competing demands. We're very lucky it's remained open and competitive. There's an alternate universe somewhere where IE6 won and you have to use ActiveX controls to enter your government ID before you can post to websites. (Oh wait, that's South Korea in this universe http://www.bbc.co.uk/news/technology-29617196)
You can write a better operating system, but unless it runs a browser and you can give it away for free it will have a very limited audience.
On the windows side, power shell.
And keep in mind, many of these scripts have been running for 10+ years. When's the last time you wrote a piece of software that worked for 10 years with minimal or no update?
No, we can't do better today. We may start something better today and enjoy it in 40 years, but for the time being, these are the tallest giants' shoulders we have.
While Perl or similar are not without many faults, at least they solve a lot of this. Think of even writing something like a JSON parser in bash... it's horrifying.
Not justifying it designed this way today and keeping a false traditionalist position to retroactively-self justify it is no worse than berating something designed at the state-of-the-art at the time with no contextual understanding of why it was done that way... it is quite possible to evolve designs within a tradition of understanding their origins rather than being a smug neophyte who self-admittedly is responding out of ones own ignorance yet choosing to still berate despite this ignorance 'because who even knows why but it seems old man'
see also: just about any incremental improvement in any long running system ever, over time, across all systems, from the dawn of time, up until today
But, having said that, I really hate how TAB is used in Makefiles. The first screen editor I used was the Rand Editor and it treated TAB as a cursor motion character. It was a handy way to move horizontally on a line by a fixed amount, something not easy to do in, say, vi. This editor converted all tabs to spaces on input and then re-inserted them on output, which mostly worked, but it could mess up some files that were fussy about tabs.
The challenge came when there were devices which operated incorrectly when presented with a TAB character or confused command data with actual data. That became the "killer" issue when computers started substituting for typewriters. Because a typist knows that if you hit the tab key the carriage would slide over to the next tab stop that was set on the platen bar, but the paper was "all spaces". When you try to emulate that behavior in a computer now the TAB becomes a semantic input into the editor "move to the next tab stop" rather than "stick a tab in" and "move to the next tab stop" could be implemented by inserting a variable number of spaces.
Computer engineers knew it was "stupid" to try an guess what you wanted with respect to tabs so they wrote a program for formatting text called 'troff' (which had similarities to the program on RSX11, TENEX, and TOPS called RUNOFF.
It is always interesting to look back though, if you had told a system designer in the 70's that they would have gigabytes of RAM and HD displays in their pocket when their grandchildren came to visit they would have told you that people always over estimate how fast change will happen.
I still prefer to indent C with tabs, and I'm not the only one. It's really not hard to handle tabs well, as a code editor or as a programmer. You can view them as whatever width you like. You can have marked to distinguish from spaces (e.g. vim's "set list"). They've been around for longer than you have so there's no good excuse for forgetting about them.
TAB was a stupid choice even in the 1970s.
Just think of the energy lost sending 4 characters instead of 1 in every uncompressed json/xml actually moving around because lazy people don't mind suppress them before sending it over the network...
— Stuart Feldman
But I'd submit that the central thesis-that it is somewhat ridiculous to adhere to many of these conventions simply because they are established-is a very good one.
We have learned so much about how to make computer systems better and almost none of it is applied to Linux. The Linux kernel in particular is the work of brilliant people using archaic methodologies and adopting conventions that have long since ceased to make sense.
The author even states that UNIX was amazing when it came out, but that doesn't mean all its ideas make sense today.
Saying that they lacked imagination is diminuitive and self-serving but certainly we can do better than those who came before us, and those who come after us will do better than us.
However good Unix might have been in the 1970s, why am I still running it on my mobile phone? How good is it today? It seems that the major argument in its favor is legacy.
(Honestly, it seems to me that most of his criticisms could be moved away from stepwise. I'm guessing most of the configuration on my phone is stored in binary files. Someone could write the object-wise commandline utils he wants, and then create the architecture to make them more efficient. We could move there. Should we?)
Unix greybeards, this is about you.
It's all churn and bloat.
* The internet (early 90s)
* Wireless networking (early 2000s)
* Highly portable, small, and lightweight computers (early 2010s)
and all the software that supports them. These "follow-up systems" are certainnly better than their predecessors, and there are literally countless more examples (no, everything shouldn't be written in asm and c).
In my experience it gets worse and worse over time.
It's not hard to see how this happened: since pretty much all computers that people normally interact with are either running Windows or a Unix-like system, it has set up a dichotomy in people's minds. When the Unix-Haters Handbook was released, there were still other operating systems which could have a plausible claim to being better, but they have all faded away, leaving only these two. And since the "Real Hackers" prefer Unix, Unix and all its decisions must be the right ones.
Unix is great, but people need to be more realistic about its shortcomings instead of mindlessly repeating mantras about "The Unix Way" without critical examination.
So all that new and advanced technology doesn't really interest me anymore. I'm looking for a 1969 Honda CL350 right now. They're still around and running fine. They're much simpler and much more maintainable. No Engine Computer. No sensors. Everything really easy to understand.
I kinda want my OS like that too. With all its warts I can keep it running.
Not true. You will be able to get an aftermarket ECU that can just ignore the sensors and run in open-loop mode. That will be exactly the same as running with carburetors: fixed fuel/air ratio that is almost always wrong. This is also the failure mode for OBDII cars - sensor failures lead to the ECU running in open-loop mode, which lowers MPG and increases emissions, which will eventually foul the catalytic converters.
> I'm looking for a 1969 Honda CL350 right now. They're still around and running fine. They're much simpler and much more maintainable.
My wife has owned a CB175 and a CB550. Both required tons of work and were maintenance nightmares. They really are piece of shit bikes when it comes to reliability when compared to most Japanese bikes from 1990 onward. The prices old Honda bikes command on the market are completely out of whack with what you get because of strong demand from both the vintage bike enthusiast and hipster demographics. I would not ride one if it was given to me for free.
Do you know about the Royal Enfield Bullet [1] from India?
It is not at all as technologically sophisticated as the bikes you mention and others, but it is a fantastic bike to ride.
They are selling it in the West, too, from some years. Originally from the Enfield company, UK, then was manufactured in India for many decades (maybe starting around WWII), as the same standard model. Then a decade or more back, the new managing director invigorated the company with better quality, newer models, higher engine capacity (cc) models (like 500 cc), etc. - though I would not be surprised that some fans prefer the old one still - maybe me too, except not ridden it enough, I rode a 250 cc Yezdi much more - also a great bike, almost maintenance free, a successor to the classic Ideal Jawa bike from Czechoslovakia, and also made in India for many years. Yezdi was stopped some years ago, last I read, but the Bullet is still going strong and even being exported, a good amount, to the West.
[1] https://en.wikipedia.org/wiki/Royal_Enfield_Bullet
https://www.google.co.in/search?q=royal+enfield+bullet
A Swiss guy, Fritz Egli (IIRC), was/is a fan and modified some of them (Bullets) over there. It was the subject of a magazine article.
I first rode a Bullet in my teens. A real thumper.
The number one flaw on these bikes in my opinion is the stock carburetors. Honda used a constant velocity type carburetor which in theory provides very smooth throttle action and is easier to ride. In reality the vacuum diaphragm is a very delicate part that frequently fails with tiny air holes that leak vacuum, causing a mismatch in the throttle input between the cylinders (twin carb two cylinder, one per cylinder). This is a similar failure mode to your modern Suzuki air pressure sensor failing.
The other pain point is the mechanical points in the ignition system. This is an area of constant fiddling with adjustment and new condensers. It's much preferred to simply replace the points system with a "modern" (1980s technology) electronic ignition system. This removes the moving parts and greatly extends the life of a tune-up.
Old bikes are super cool and the 350 platform is a fantastic one but even back then bikes had "high tech" parts that did not age well and gaps where better tech had not been invented. The great thing about the 350 platform is that due to its popularity people are still coming up with solutions. In this way a Honda 350 is similar to Unix.
Not Linux, surely. That would be more like the Suzuki, but it used to be an older bike so they left the carburetor in there and next year someone will add an electric motor too.
This was not a fancy engine. It was not a particularly powerful engine. Not a single thing on it or about it was exceptional. Because it was easy to work on and with add-ons and modifications it could be coaxed into doing things it wasn't really meant to do. It was just solid and got out of your way. So engineers and hobbyists tinkered and tinkered and got something over 2.5x the original horsepower out of the thing.
Unix commands might be solid, but the 'get out of your way' bit is what troubles me. People will lament otherwise good engines that are hard to work on because of a design flaw or the way they're laid out. Unix is falling down here. It's just that everyone else is at least as bad. But a guy can dream.
When I thought I wanted a bike, I looked around for a Condor A350.
Hence UNIX (of a sort) is actually quite popular on both iPhones and Android - but not in ways accessible to the user. Even if you do have to replace Android devices every few years because the manufacturer won't maintain them either.
Spare parts are easy to find and there are a lot of aftermarket parts.
The downside is that the bike is basically a late '80's bike. 40ish horsepower, poor fuel economy, weak brakes, poor handling, etc...
But, like lots of people say, it's more fun to drive a slow bike fast than a fast bike slowly. I love it.
That's why we have both "ls" and "find", even though they do the same thing conceptually. "ps" has column output, but the way it formats, sorts, selects etc. columns is reinvented and not transferable to other tools such as "lsof" and "netstat", not to mention "top", which of course is just a self-updating "ps". Every tool invents filtering, sorting, formatting etc. in their own unique way. The various flags are not translatable to other tools.
I've never used Powershell, but it seems to get one thing right that Unix never bothered to. Tools should emit data, or consume data, or display data, but tools should never do all three, because they will invariably reimplement something poorly that has been done elsewhere.
The right way to do this is to pick some reasonable text-based structured interchange format - s-exprs, JSON, whatever. Actually, it wouldn't be a bad thing to have a binary protocol as well, so long as all that complexity is negotiable, and is implemented by the standard library.
(That bug, and option, was what finally convinced one old-time shop to swap in SSH for RSH for such tasks, after I identified the problem and the solution.)
A better example would be "one tools" that do many things. Rob Pike suggested Perl, though IMO that's kind of foolish: Perl is Yet Another Programming Language, and programming languages, rather by design, tend to do multiple things (not always, or not always easily: see sed as an example of both caveats -- text replacement or branching code).
Other examples are generally nexus points of complex interactions. Editors (most especially Emacs). Email management (communications protocols, format protocols, ties to DNS, anti-spam, filtering, remote-access (POP and IMAP), extensions, authentication (at various levels), encryption (at various levels). The problem domain is complex, so the tools are complex.
The tool you're very likely using right now to read this would be another case in point: the Web browser. Not just for CERN any more.
I don't get what you're trying to say.
"ls" lists files in a specific path while "find" searches for objects in a file system that match a specific query.
Are you sure you're not confused and somehow assuming that "does just one thing well" should mean "another tool must not accomplish anything that might be remotely achievable by misusing another tool"?
http://web.mit.edu/~simsong/www/ugh.pdf
It even has an anti-foreword by Dennis Ritchie which kinda reminds me of the Metropolitan Police spokesman's blurb on the back of Banksy's book.
BTW, it starts off with an anonymous quote that I've never heard, Two of the most famous products of Berkeley are LSD and Unix. Unix is of course from Bell Labs. And anyone who knew anything would have said instead, Two of the most famous products of Berkeley are LSD and BSD which at least would have been funny if still inaccurate. Anyways, it seems like a fun rant of a book which I'd never heard of.
The above point about not getting it can be applied to Linux as well. Lessons learned elsewhere are stupid until Linus finally understands them and then they're obvious.
From: dmr@plan9.research.att.com Date: Tue, 15 Mar 1994 00:38:07 EST Subject: anti-foreword
To the contributers to this book:
I have succumbed to the temptation you offered in your preface: I do write you off as envious malcontents and romantic keepers of memories. The systems you remember so fondly (TOPS-20, ITS, Multics, Lisp Machine, Cedar/Mesa, the Dorado) are not just out to pasture, they are fertilizing it from below.
Your judgments are not keen, they are intoxicated by metaphor. In the Preface you suffer first from heat, lice, and malnourishment, then become prisoners in a Gulag. In Chapter 1 you are in turn infected by a virus, racked by drug addiction, and addled by puffiness of the genome.
Yet your prison without coherent design continues to imprison you. How can this be, if it has no strong places? The rational prisoner exploits the weak places, creates order from chaos: instead, collectives like the FSF vindicate their jailers by building cells almost com- patible with the existing ones, albeit with more features. The journalist with three undergraduate degrees from MIT, the researcher at Microsoft, and the senior scientist at Apple might volunteer a few words about the regulations of the prisons to which they have been transferred.
Your sense of the possible is in no sense pure: sometimes you want the same thing you have, but wish you had done it yourselves; other times you want something different, but can't seem to get people to use it; sometimes one wonders why you just don't shut up and tell people to buy a PC with Windows or a Mac. No Gulag or lice, just a future whose intellectual tone and interaction style is set by Sonic the Hedgehog. You claim to seek progress, but you succeed mainly in whining.
Here is my metaphor: your book is a pudding stuffed with apposite observations, many well-conceived. Like excrement, it contains enough undigested nuggets of nutrition to sustain life for some. But it is not a tasty pie: it reeks too much of contempt and of envy. Bon appetit!
Oh for the days when 750kB was considered "massive" for a binary.
> Here is my metaphor: your book is a pudding stuffed with apposite observations, many well-conceived. Like excrement, it contains enough undigested nuggets of nutrition to sustain life for some. But it is not a tasty pie: it reeks too much of contempt and of envy.
> Bon appetit!
Pretty good.
Research Unix, yes. But the culture of unix is much bigger than some single copyright holder... this is why the entire thing is plain stupid. Study the levenez family tree. Read the archived Usenet posts and get a copy of the Cuckoos Egg.
BSD IS UNIX.
I think you're suffering from selection bias.
I really wish undergraduate Software Engineering programs included a course that was a survey of operating systems, where you'd write the same program (that did a lot of IPC) on, say, base WinNT (kernel objects with ACLs!); base Darwin (Mach ports!); a unikernel framework like MirageOS; something realtime like QNX; the Java Card platform for smart-cards; and so forth. Maybe even include something novel, like http://genode.org.
I eventually got very surprised that not all universities do that.
Hey, that sounds like Go's line. Oh wait ..
As a developer, it was a bit painful, but the whole file versioning was handy sometimes.
Probably the biggest one is that UNIX is, at bottom, a terminal-oriented multi-user time sharing system. This maps badly to desktop, mobile, and server systems. The protection model is a mismatch for all those purposes. (Programs have the authority of the user. Not so good today as in the 1970s.) The administration model also matches badly. Vast amounts of superstructure have been built to get around that mismatch. (Hello, containers, virtualization, etc.) Interprocess communication came late to UNIX/Linux, and it's still not a core component. (The one-way pipe mindset is too deeply ingrained in the UNIX world.)
POSIX is more like the extended runtime that C needs to be portable outside UNIX walls, which isn't being fully implemented on iOS and Android.
http://www.cs.columbia.edu/~vatlidak/resources/POSIXmagazine...
Yes, you have to use their APIs in order to write graphical programs, but the same is true on any OS with a GUI system.
It's possible to write iOS apps in pure C if you want to. Sure, that'd be a pain, but it's possible. Less painful and actually decently reasonable would be to write all the GUI-specific stuff in Objective-C and any other logic in pure POSIX-conforming C or C++, since you can mix all those languages freely in a project.
All programming languages, other than Assembly have runtimes.
From the point of view of compiler design a runtime is everything required to support a programming language execution.
On C's case, calling main (), floating point emulation, initializing globals, parallel code execution, thread locals.
I disagree, I think this is still the sweet spot between security and utility. Users have been trained to just click approve on any privilege escalation dialog.
From the description of Combex (http://www.skyhunter.com/marcs/capabilityIntro/):
> Suppose you were running a capability-secure operation system, or that your mail system was written in a capability-secure programming language. In either case, each time an executable program in your email executed, each time it needed a capability, you the user would be asked whether to grant that capability or not. So Melissa, upon starting up, would first find itself required to ask you, "Can I read your address book?" Since you received the message from a trusted friend, perhaps you would say yes - neither Melissa nor anything else can hurt you just by reading the file. But this would be an unusual request from an email message, and should reasonably set you on guard.
In reality, users will get sick of being prompted every 30 seconds and learn to automatically approve every request. Capability security works well in theory, but I've never seen an implementation that works well in practice.
I think the next step is a capability runtime OS with a kernel personality for Linux for backwards compatibility. Sort of the converse of what we're doing right now.
I feel so privileged to read this random guy's blog, and it's terrific that he eschews inflating his ego so well.
The author is taking the time to post his thoughts on a private blog. If you're not happy with the level of rigor, then don't read it, don't share it, and don't believe it. But no, the author has no responsibility to provide you with a comprehensive list of citations and references.
What I don't like about that attitude is that it suggests that the blogpost should be excused from criticism of its rigor. But any assertion and opinion by anyone can and should be considered open to criticism.
> "But no, the author has no responsibility to provide you with a comprehensive list of citations and references."
Of course he is not responsible. But if someone expects or wants people to be convinced of their argument, then prefacing with his rude dismissal does not help.
A better way than "This article was written hastily, and I don’t want to further improve it. You’re lucky I wrote it." without the unnecessary rudeness and ego boosting could be more along your lines: "This article was written hastily and shouldn't be subject to PhD-thesis-level scrutiny, so I've posted without any expectation that I will improve it or respond to criticism."
I'm not saying it has to be PhD-thesis-level, but if proven wrong you are morally obligated to update the information or remove it. Otherwise you are spreading false knowledge. This doesn't apply to opinions ofc (the bulk of private blogging).
Screw entitled users though, they should fix it themselves :)
I was envisioning people nit-picking the lack of formality, when it was merely an enthusiastic, from-the-hip post.
A day later I realized that wasn't the best way to start things off with, and toned it down. But I get where he's coming from. And I'm grateful he took the time to write it. It was an interesting read.
You can just start the countdown until all the unix greybeards come out and slams them with why it's so wrong and how to do it with some obscure bash feature.
You wouldn't consider it to be immature to not do the right thing just because it was communicated in an unpleasant way that a 'self-respecting person' should have disdain for? I'm not talking about this particular article or defending immaturity, - I just don't see how your position is any better; more so it's just as unhumble as the 'rockstar position' since you would apparently be willing to write worse code just to avoid learning from someone that's an asshole.
>How to touch all files in foo (and its subfolders)?
find foo -print0 | xargs -0 touch
find foo -exec touch {} \;
> The provocative tone has been used just to attract your attention.
Rob Pike 2004, https://interviews.slashdot.org/story/04/10/18/1153211/rob-p...
The problem here is that there aren't a lot of alternatives. You could use Windows, which is like listening only to music by MC Hammer, or you could use a Mac, which is like listening only to music by Duran Duran.
Because of software/backwards compatibility concerns, and how dependent everything is on the underlying OS, it's really hard to change anything in the OS, especially the fundamental design. It'd be nice to make a clean-sheet new OS, but good luck getting anyone to adopt it: look at how well Plan9 and BeOS fared.
You say that like it were a bad thing.
Whereas Mac OS, Windows, iOS, Android, ChromeOS have moved into more productive language runtimes, with rich frameworks, improving safety across OS layers, even if they have a few bumps along the way.
In view of that, it's only natural that big segments of software beyond the OS like word processors are also stagnant. We don't actually need a diverse marketplace of competing word processing ideas anymore... the problem is fundamentally solved as far as the public is concerned. It's not as exciting for software developers, but it's totally natural for it to happen.
Now if you do want page layout, I think the people who like to do that use something different — not Word. So I don't think they've really cornered any market, unless the market is people who want to use Word.
Try and read it this way:
"We really are using a 1970s era operating system ..."
There are two types of languages, the ones everyone complains about, and the ones nobody uses.
Passing text streams around is a horrible idea because now each program has to have its own, half-assed shotgun parser and generator, and you have to glue programs together with your own, user-provided, half-assed shotgun parsers, i.e. calls to awk, sed, etc.
Think of it this way: if, per Unix Philosophy (points (1) and (2) of your summary), programs are kind of like function calls, and your OS is kind of like the running image, then (3) makes you programming with a dynamic, completely untyped language which forces each function to accept and return a single parameter that's just a string blob. No other data structures allowed.
I kind of understand how it is people got used to it and don't see a problem anymore (Stockholm syndrome). What shocked me was learning that back before UNIX they already knew how to do it better, but UNIX just ignored it.
Right. The title should have been reflective of that "Various idiocies Unix has accumulated to this day" but since the article mentions Unix Philosophy, my point is that the article should have criticised the philosophy and not the practice.
> "Passing text streams around is a horrible idea because now each program has to have its own, half-assed shotgun parser and generator, and you have to glue programs together with your own, user-provided, half-assed shotgun parsers, i.e. calls to awk, sed, etc."
But this has actually proved to be very useful as it provided a standard medium of communication between programs that is both human readable and computer understandable. And ahead of its time since it automatically takes advance of multiprocessor systems, without having to rewrite the individual components to be multi-threaded.
> "(3) makes you programming with a dynamic, completely untyped language which forces each function to accept and return a single parameter that's just a string blob. No other data structures allowed."
That may be a performance downside in some cases, but the benefit of having a standard universally-agreeable input and output format is the time it saves Unix operators who can quickly pipe programs together. That saves more total human time than gained from potential performance benefits.
It wasn't ahead of its time. By the time Unix was created, people were already aware of the benefits of structured data.
> it automatically takes advance of multiprocessor systems, without having to rewrite the individual components to be multi-threaded.
That's orthogonal to the issue. The simple solution to Unix problems would be to put a standard parser for JSON/SEXP/whatever into libc or OS libraries and have people use it for stdin/stdout communication. This can still take advantage of multiprocessor systems and whatnot, with an added benefit of program authors not having to each write their own buggy parser anymore.
> but the benefit of having a standard universally-agreeable input and output format is the time it saves Unix operators who can quickly pipe programs together. That saves more total human time than gained from potential performance benefits.
I'd say it's exactly the opposite. Unstructured text is not an universally-agreeable format. In fact, it's non-agreeable, since anyone can output anything however they like (and they do), and as a user you're forced to transform data from one program into another via more ad-hoc parsers, usually written in form of sed, awk or Perl invocations. You lose time doing that, each of those parsing steps introduces vulnerabilities, and the whole thing will eventually fall apart anyway because of million reasons that can fuck up the output of Unix commands, including things like your system distribution and your locale settings.
As an example of what I'm talking about, imagine that your "ls" invocation would return a list of named rows in some structured format, instead of an ASCII table. E.g.
((:columns :type :permissions :no-links :owner :group :size :modification-time :name)
(:data
(:directory 775 8 temporal temporal 4096 1488506415 ".git")
(:file 664 1 temporal temporal 4 1488506415 ".gitignore")
...
(:file 755 1 temporal temporal 69337136 1488506415 "hju")))
ls | filter ':modification-time < 1 month ago' | cp --to '/home/otheruser/oldfiles/'
find :name LIKE ".git%" | select (:name :permissions) | format-list > git_perms_audit.log
BTW. This is exactly what PowerShell does (except it sends .NET objects), which is why it's awesome.
Except it is completely unusable for network applications because the error handling model is broken (exit status? stderr? signals? good luck figuring out which process errored out in a long pipe chain) and it is almost impossible to get the parsing, escaping, interpolation, and command line arguments right. People very quickly discovered that CGI Perl with system/backticks was a very insecure and fragile way to write web applications and moved to the AOLServer model of a single process that loads libraries.
I think this is great. There's slightly more principled ways to do it, but having to convert everything to one single format at the end of the day keeps you humble.
Let's go back to the previous decade's Hacker News:
All of UNIX makes perfect sense if you are using UNIX for UNIX.
If you're doing other things, like abstracting to "folders" and so on ... I am open minded and can see where it starts to fall apart a bit.
But I use UNIX for the sake of UNIX ... I am interested specifically in doing UNIX things. It works great for that.
This could be either a no-true-Scotsman, or a tautology. To solve this, you'd need to specify what UNIX is good at.
It's even worse!
I am saying that working in terminals, with strings of text and non-binary-format config files ... and all of the tools built around that ... is an end in itself.
Every single "broken" example in the OP is something that I find non-remarkable and, in fact, makes perfect sense to me.
Further, there is an immense value in GUI based systems: discoverability. On a GUI, you can learn how to use a program without ever consulting a manual just by inspecting your screen. This addition is what brought the computer to the masses.
Finally, the terminal model of UNIX is just horrible. The hacks-on-top-of-hacks that are needed to turn the equivalent of a line-printer into something like nCurses or tMux are horrible. The current terminal is like this purely because of legacy. If you'd design a system for "working in terminals, with strings of text and non-binary-format config files" from the bottom up, it would look totally different. Sadly, getting it to work with existing software would be a total nightmare.
All that being said, UNIX still has the better terminal (though I hear good things about powershell). Certainly, it is the best system for "working in terminals, with strings of text and non-binary-format config files". Though competition is sparse (windows, and maybe mac, depending on whether you consider it to still be unix or not).
Often though it just reflects how recently you started using computers (more than MS vs Unix). I predominantly work in Windows but I call them directories but then I started with MS-DOS (and you type "dir" for a listing, right?).
If you started using computers with a GUI (and that's where you're most comfortable) then you might think of them as folders because on most platforms that's what the icon looks like.
I think the "folder" metaphor starts to break down a bit as soon as you're dealing with remote systems. Navigating to a parent "folder" on a remote FTP sounds weird to me, and it's telling that when browsing a FTP in a web browser even Internet Explorer 11 refers to directories - e.g. "Up to higher level directory" not higher level folder.
> it just reflects how recently you started using computers
> sounds weird to me
Are those actual arguments? I am not sure the parent poster asked what you found "weird" or wanted you to imply you use computers for a long time.
For what is worth, I'd still prefer "mount point" or "access point" to both "directory" or "folder" when talking about remote systems. So I am not bashing you -- it's just that your comment is full of subjectivity.
You do understand that the article was written in different language (russian), right? I kinda think this makes your whole argument broken.
1) Text is for humans, and is generally incomprehensible to machines. Encodings, arbitrary config file formats, terminals, etc, are all piles of thoughtless one-off hacks. It's a horrible substrate for compositing software functionality, through either pipes or files.
2) Hierarchical directory structures quickly become insufficient.
3) The filesystem permission model is way too coarse grained.
4) C is a systems/driver/low-level language, not a reasonable user-level application or utility language.
Actually, I'd take plain text conf over Win10 registry any day also. How do you transfer your (arbitrary) program settings from one computer to another? I can tell you how on Unix. How would you on windows?
2. / based filesystems are head and shoulders better than Windows.
Why should moving a directory from my internal hard drive to a SSD over USB change my location?
On Unix I can keep my home directory (or Program Files) on a NFS share, SMB share or SSD harddrive.
Can I do the same on Windows?
3. It is, which is why SELinux was invented. But that's to hard, so no one uses it.
4. All major OSs (Both Unix/Linux and Windows) are C or C++ based.
But here's something interesting: in the 90s, it was considered "advanced" to have the GUI an inherent part of the OS, rather than being just another program. Windows and Plan9 did that.
Yet, it turned out that admining a system without a 1st class command line is a pain, so windows is rolling back with Power shell.
Maybe Windows will one day make their Server headless, and where you can do full sysadmining through SSH
Your personal registry settings are stored in %USERPROFILE%\ntuser.dat, and global registry trees are various files under %SystemRoot\System32\Config.
Actually, in many ways, Windows makes its organization system much easier to find profile data to copy than Unix-based systems. Most programs install configurations either in their install folder (C:\Program Files\<Product Name>) or in per-user data (C:\Users\<user>\Application Data\<Product Name>). In Unix, you might have a mix between dot folders (with some things choosing to use .<name> while others might settle for .config/<name>), as well as having to guess if various config files are under /etc/<product>, /usr/lib/<product>, /usr/share/<product>, or maybe even weird places like somewhere under /var or /opt.
> On Unix I can keep my home directory (or Program Files) on a NFS share, SMB share or SSD harddrive. Can I do the same on Windows?
Yep, you can remap where your user's home profile is. You can even distinguish between files that should be replicated across network shares and files that should not be replicated in per-user storage (AppData\Roaming versus AppData\Local).
See my other response below about the text issue. It's not about 1 program having a text file, it's about managing all text files on your system that sucks, especially when you want them to interoperate in terms of data or automation, or even have some sort of consistency in expectation between them in human terms. The term "Text" implies data that is unspecified and unpredictable; in encoding, syntax and semantics; across various instances of it.
Export/import the relevant registry keys, just as you copy/overwrite the relevant config files. If you can't find the reg keys, what makes you think you can find the config files?
You don't even need to do anything for your roaming user profile to follow you on a windows domain! I couldn't tell you how to accomplish the same on Unix.
> Can I do the same on Windows?
I routinely (ab)use mklink to keep the path identical despite moving things between disks or onto remote shares. Different subdirs within C:\Program Files\ are often on separate disks - just copy, paste, and mklink. I haven't run into an application that complains (although I have had installation scripts botch things on rare occasion by deleting my mklinks.)
You can also do unix style mount-a-disk-in-an-empty-directory if that's your jam, and have e.g. C:\SSD. For the GUI, see the "Disk Management" tool - RMB the partition, "Change Drive Letter and Paths...", "Add...", (*) "Mount in the following empty NTFS folder:". Powershell can use WMI: https://learn-powershell.net/2012/08/10/locating-mount-point...
Computers are for humans. I like my configurations to be accessible to me, as well as my logs.
It's a fundamental, unstandardized mess.
From the perspective of just a single program, then fine, it does what it needs to and its individual text formats are human-understandable, and its code is quickly banged together to do what it needs with it.
From an ecosystem perspective, it's a complete failure of usability and _interoperability_, which is the entire point of Unix philosophy in the first place!
Humans aren't the only ones reading & writing config files: Software needs to do it, too, be it applications or automation scripts. Again, text is a horrible substrate for compositing software functionality.
(More people would probably have read the Unix Haters Handbook if it was as pithy as this.)
1) Standardize on some structured text serialization format (I like YAML for this)
2) Write a new shell
Both of these things are compatible with the Unix Philosophy™, and thus said Philosophy is nowhere near collapse. Rusty around the edges, sure, and maybe with some asbestos in the ceiling tiles, but certainly renovatable.
The philosophy is already prevalent in the world of "microservices"; an application is split into a whole bunch of independent (usually containerized) programs communicating via something like JSON over HTTP.
Great idea! We should use XML for that...
So yeah, any of that would be a much better idea than unstructured text, and yes, you can serialize all those use cases into trees. I'd steer away from XML for sake of efficiency and human-readability though.
The terminal is responsible for hacks upon hacks: Colors, ncurses, signals, and whatnot.
1) This is an appealing idea, but my claim is that there's no single serialization format that will work. (Or if there is one, it has yet to be invented.) More detail here:
https://www.reddit.com/r/oilshell/comments/5x5rgg/pipes_in_p...
There's nothing stopping anyone from using structured data over pipes, but I think it's a mistake to assume there will be or needs to be a "standard".
3) I agree that JSON over HTTP is very much in the vein of Unix. The REST architecture has a very large overlap with the Unix philosophy -- in particular, everything is a hierarchical namespace, and you have a limited number of verbs (GET / POST vs. read() / write() ).
Then I'm not sure how the problem would be solved. The main reason why "everything should be plaintext" is problematic (aside from being inefficient storage-wise) is that there's no "standard" format. My interpretation of the article's criticisms is that overreliance on tools like awk is the problem, not the solution.
Hence, the recommendation to just standardize on YAML (or some stricter subset thereof). If unstructured data is really needed in the pipeline, then it can easily be encapsulated in an ordinary YAML document. This would unify the strengths of the Unix way (ease of human inspection) and the PowerShell way (ease of plugging arbitrary tools together without needing to stick a bunch of text filters all over the place).
Note that this philosophy covers many concepts. These discussions often mention the modularity and composition rules, but the other parts of the philosophy are also important.
See "The Art Of Unix Programming" for a full explanation of the philosophy.
Suckless and cat-v.org would disagree. I'd also disagree since I'm a huge fan of plan9port.
What is missing in many cases is a concepts guide, explaining the key ideas, how to combine things, and what's possible in various subject areas.
For GUI programs, menus / toolbars used to be the concept guide: what they show is what's possible, and they offer context help. This is why a GUI feels friendly. It sucks at composability, though. Current mobile interfaces, unfortunately, tend to lack this.
If tiny GUI-oriented programs were easy to compose, had an easy way to save the composed state, and a number of daily-use programs bundled with an OS came in this form, providing example and reference, many people would consider following the suit, I suppose.
[1]: http://softwareengineering.stackexchange.com/questions/61814...
This simple fact seems like the key to getting the masses into computing. For something like 6 years (say 12-18) GUIs were the way I interacted with computers. Need to do something and learn about it? Go and explore the UI until you find the option. If the option has a shortcut printed, you will remember it eventually.
Sadly, GUI design is a quite separate discipline from software design. This means much open source software is missing GUIs. Those who write the software aren't always GUI designers. This also creates the mismatch between composing software and composing GUIs. As they are different disciplines, what it means to combine them means different things.
A decent stopgap is massive frameworking and standardization on GUI to make it easier for devs to get a GUI. To get the really good stuff, commercial entities have the best position. They need their stuff to be usable by everyone, and this finances the hiring of GUI people.
There is the rare gen of a developer that can also do GUI right, but that only has value in the case of small projects. When projects grow, unless all devs have the GUI knack, you're gonna need some dedicated GUI people.
It would be great if we could get more GUI-oriented people into open-source stuff but it seems like they aren't as attracted to open-source as devs are. It might be because devs can be at the ground floor of a project, and GUI, almost by neccesity, comes later.
"The Art of Unix Programming"
http://www.catb.org/esr/writings/taoup/html/
It's not perfect - I'd love to see a guide with more practical examples - but it does do a good job covering the basic philosophy and some of the history of why certain design decisions were made.
What I mean to say is that while there has been a lot of initial inertia from the previous technologies that make it hard to change, it's also true that the new technologies have failed to make large enough gains to warrant the pains of changing - this is precisely the demise of Plan9: it's not that it wasn't better it just that it wasn't better enough to warrant the huge expense of replacing old, working systems.
This is the OS equivalent of a Call of Duty teenage online player on XBox live.
I guess I am lucky he wrote this for a pleb like me.
https://lwn.net/Articles/411845/
The whole series is worth a read, especially part 3 'unfixable desgins' which talks about Signals and the Unix permission model.
Whenever someone tries to critique UNIX they always make up these nonsensical problems.
No UNIX user would intentionally name a file with a forward slash, a space, a semicolon, etc.
But I will play along.
To find these files, a number of ideas come to mind.
Maybe the easiest would be to use mtree to make a file specification and then search the specification.
Something like
mtree -cp /foo |exec mtree -C
If there are so many files that these specifications become large enough to cause problems with UNIX utilities, I import them into a database like kdb.
Is there a way to hide files from mtree? Maybe.
But at least with the UNIX concept for a system, the user can look at the source and see how it works.
The truth is, UNIX is not so impressive.
It is only the fact that the alternative systems for doing the types of things one does with UNIX have always been inferior/rubbish in the opinion of a certain set of informed users.
It is this contrast that makes UNIX seem impressive.
As usual, the author makes no suggestion or detailed comparison of any other alternative that can be used for doing the tasks one does with UNIX.
> How can we recursively find all the files with \ name in a folder foo? The correct answer is: find foo -name '\\\\'
No, the correct answer:
% find foo -name '\\'
foo/\
The single quotes prevent the shell from performing escaping. Find does need escaping, and that's the only layer that does, so the correct answer has two backslashes.
It's valuable to challenge assumptions held in a community (to avoid the Normalization of deviance in software: how broken practices become standard[2]). It's more valuable to suggest improvements, and most valuable to improve it.
[0]: http://web.mit.edu/~simsong/www/ugh.pdf [1]: https://www.dreamsongs.com/WorseIsBetter.html [2]: http://danluu.com/wat/
If the system as designed accepts almost literally any characters in pathnames, it doesn't make a lot of sense to complain about people using "naughty" (troublesome) characters in pathnames they create.
It would have made a heck of a lot more sense to have disallowed stupid characters (all white space and control characters at a minimum) from the outset. However, changing it at this point is pretty impractical.
Of all the criticisms of UNIX, the singular one that strikes me as valid is the way that it accepts unquestioningly insane characters in pathnames. I would also maintain that case sensitivity in pathnames and a lot of other places is also a human interface error. It is arguable, and a matter of taste, but I find it obtuse. You should be able to "say" pathnames. The necessity for circumlocutions like "Big-M makefile" is offensive.
I'm one of the biggest fans there is of UNIX and C and the Bourne shell and derivatives, but recognition of weaknesses and flaws in those you love is not weakness; it is wisdom.
In fairness, though, allowing non-ASCII characters is what enabled the (mostly seamless) transition to UTF-8 filenames.
"So don't do it."
The problem with filenames is just a symptom of the biggest problem of UNIX conventions - passing around unstructured text. Filenames should have one well-defined format (AFAIR kernel allows pretty much anything but the NULL character). That's it. For most applications, filenames should be opaque data blobs compared for binary equality. But because we're passing around unstructured text, each program has to parse, reparse, and concatenate strings via ad-hoc, half-assed shotgun parsers. Each program does it slightly differently, hence the mess.
As long as everyone recognized that putting certain characters in pathnames was counter-productive, things worked fine. Nobody ever dreamed of putting a space character in a filename when they all came from a CLI background. When barbarians came from Mac/Windows to UNIX, they lacked this background, and there went the neighborhood.
I remember being stunned when I first encountered two periods in one filename! But it only took me a few seconds to grasp the fact that period was just another character, "suffixes" were just conventions, and it all made perfect sense. OTOH, the first time somebody showed me a file named "-fr\ \*" and suggested I delete it, I got one of my first disillusionments.
P.S. - "/" is a character which is also "special" in pathnames. Actually, the particular FS may make other exceptions which are not globally enforced by the kernel. ZFS has several (such as "@").
There is an excellent discussion of the topic[0]. I find it utterly definitive in the way it relentlessly shows how you can't "fix" this issue completely any other way than by ruling out the bad characters by making the kernel disallow them.
[0] https://www.dwheeler.com/essays/fixing-unix-linux-filenames....
- Dirty hacks in UNIX started to arise when UNIX was released, and it was long before Windows came to the scene, I guess there wasn’t even Microsoft DOS at the time (I guess and I don’t bother to check, so check it yourself). At least he acknowledges that he's being incredibly lazy, and he shows the glimmer of an understanding as to why some of the things mentioned later happened: because Unix is from the early 70s, which were a very different time in computing & culture.
- Almost at the very beginning, there was no /usr folder in UNIX. All binaries were located in /bin and /sbin. /usr was the place for user home directories (there was no /home). Putting /home on a separate partition remains a pretty common thing to this day because users will tend to have greater storage requirements than just the root. /usr/bin and the like are the result of people realizing that this secondary larger disk is an acceptable place to put binaries and other files that aren't needed at bootup.
- In other words, if you’ve captured Ctrl+C from the user’s input, then the operating system, instead of just calling your handler, will interrupt the syscall that was running before and return EINTR error code from the kernel. That's not the kind of interrupt they're talking about.
- I’ve read somewhere that the cp command is called cp not because of copy but because UNIX was developed with the use of terminals that output characters very slowly. Yep, terminals that print on paper are pretty slow, as are 300 baud modems. I'm absolutely crushed I had to learn that 'cp' means 'copy'--it took hours to beat that into my head, and the thousands of keystrokes I've saved over the years are a small comfort (except to my rsi-crippled hands)
- The names of UNIX utilities is another story. For example, grep comes from command g/re/p in the ed text editor. Well, cat comes from concatenation. I hope you already knew it. To top it all up, vmlinuz — gZipped LINUx with Virtual Memory support. 'cat' comes from 'catenate', in fact. What would you name 'grep' instead? "searchregexandprint"?
- at least the main website of C that would be the main entry point for all beginners and would contain not only documentation but also a brief manual on installing C tools on any platform, as well as a manual on creating a simple project in C, and would also contain a user-friendly list of C packages This is one of the most ridiculous ones. You're talking about a programming language defined in the 70s, for Christ's sake. Lot of websites created in the 70s? There is a document with a good introduction to C, project examples, etc. and it's call The C Programming Language, a book by K&R. When Kernighan made another language a few years ago, yeah, he made a website for it--golang.org, it's one of the best project sites I've seen.
The article points out some legit problems in Unix, but even leaving aside the author's ESL challenges it's poorly-written, poorly thought-out, and poorly-defended.
- make's TAB "problem" as very first argument is not very convincing.
- The citations to back up the claim that a (possibly binary) registry database was better than small text files are just not backing up. There is nothing to defend a registry there. The quote is about fsync semantics which has nothing to do with registry. Btw. in my perception it's a widely accepted fact that a big-pile-of-crap database is a bad idea. And oh, I haven't ever heard of any problem with passwd/group/shadow/gshadow being text files. And if there were, the access method is actually abstracted away, it's easy to switch backends to something else (NSS). (there is a problem with these files though -- they are denormalized, and not all fields have clear meaning and some programs interpret some fields in weird ways.).
- Zombie processes. What's the problem there? They are just like file handles. Handles have to be closed before they are garbage collected. The actual problem is that you can't really "open" and "close" processes, only spawn new childs and the resulting hierarchy is not typically desired.
- "We call touch in the loop! This means there is a new process for each file. This is extremely inefficient." Yeah and why exactly is the shell to blame that you use touch in a loop? (apart from the fact that it's almost certainly not a problem).
Could go on but have to leave...
> We call touch in the loop here (I do know that the code can be re-written with xargs, so that touch is called only once; but let’s forget about it for now, okay?)...This means there is a new process for each file. This is extremely inefficient. Code in any other programming language will work faster than this one.
WHAT THE HELL??? Did he just criticize Unix/POSIX for its inefficiency after willfully choosing a less efficient way?
He wrote
find foo -print0 | while IFS="" read -rd "" A; do touch -- "$A"; done
find foo -print0 | xargs -0 touch --
Yeah, I found that one an especially weird gripe. Grepping was a new thing, so we needed a word for it. 'Grep' is short, easy to say and type, and relatively hard to confuse with similar words in the domain. Works for me.
I can unfortunately imagine a modern startup implementing it, and shudder at potential names my imagination is coming up with... Searchlr, the best way to search text! ReadMonkey, your personal pattern recognizer! I'll stop now.
The author cites to this post by Rob Landley – http://lists.busybox.net/pipermail/busybox/2010-December/074...
While I can't independently confirm Rob's claims, and he doesn't provide any citations, I do find them very believable – /usr was invented at Bell Labs because they were running out of space on their puny 1970s hard disks. (And an RK05 was small even by 1970s standards – the IBM 2314 mainframe hard disk, released in 1965, had a 30MB capacity; the IBM 3330, released in 1970, stored 100MB – of course, these disks would have cost a heck of a lot more than an RK05, and were likely not feasible for the UNIX team given their budget.)
If they had bigger disks (or the facility to make multiple smaller disks appear like one big disk) – it is less likely they would have split the operating system itself across two disks (/ and /usr). (Using separate disks for the OS vs user data was more likely even with bigger disks since that was common practice on systems at the time.)
(Some other operating systems from the same time period already had some ability to make multiple disks appear like one big disk. For example, OS/360 has the concept of a "catalog", which is a directory mapping file names to disk volume names; this means you can move individual files between disks without changing the names by which users access them. In their quest for simplicity, Thompson and Ritchie and co decided to omit such a feature from UNIX.)
Lisp, Smalltalk, Mesa, Pilot OS and Xerox Development Environment are from the early 70s as well (Lisp even earlier).
I suspect the author of this site have contributed to the linux haters handbook.
Some quotes:
“Linux printing was designed and implemented by people working to preserve the rainforest by making it utterly impossible to consume paper.” – Athas
“ALSA is like the emperors new clothes. It never works, but people say it’s because you’re a noob.”
“Object-oriented programming is an exceptionally bad idea which could only have originated in California.” – Edsger Dijkstra
“[firefox] is always doing something, even if it’s just calculating the opportune moment to crash inexplicably” – kfx
....
1969 < 1981.
Heck, in 1969 there wasn't even RT-11 that CP/M was modelled after.
There was a brand new OS/8 that RT-11 was modelled after.
If we scan this thread we could come up with a list of issues/problems with UNIX. Human inclination is to instantly look for each solution as each problem presents itself. Eg. "Text as an interface is unstructured" -> "Use other formats". The end result becomes a lot of feature creep, adding layers over the OS to hide the usability issues of the past. Then, a new user comes along and wonders, "How did it all get so messy?"
The ideation-style approach to innovation is NOT to leap straight to solutions. Avoid analyzing the problems immediately. Don't criticize other people's existing thoughts.
Instead we use our creativity to add to the list of problems. We build it up even more. Add tangential issues that may be not just UNIX related problems. Add future issues. Add past issues. You keep adding until you exhaust all the avenues. You don't want to block your thoughts or anyone else's. If anything, the previous problem someone raises should inspire your next one.
Once you have drained out all the issues and you can't squeeze out any more complaints, then you can take a step back and look at all the problems as a whole. You group them into categories that have common themes. You try to generalize and re-express them in vaguer terms.
After you have your themes, you can think about making a list of solutions. Again you don't want to be critical about the feasibility of a solution. You just want to build a list of different solutions. Each solution should inspire a new different solution. There are mental exercises that you can do to inspire tangential thoughts - word games etc.
When you have a giant list of solutions, then you categorize again. Those categories are the start point to building something innovative.
The alternative is just another iteration of what already exists.
My own thoughts on this are that I seem to need "power-user" abilities way too much. My field is actually Hardware. Something is wrong if I am jumping through a lot of hoops outside of my specialty just to get work done. It's not just UNIX either. My friends and family often call on me to handle their Windows or Mac issues because I am the closest thing to an expert they know. They shouldn't have to.
The rest, well, its a helpful check on over enthusiasm to be familiar with the shortcomings of UNIX. By dint of its popularity, diversity and long lineage, UNIX and its derivatives are particularly rich in warts. Yet, I think it must be true of all real-world operating systems with a long life that cruft, workarounds and ossified bad ideas accumulate.
However, for all the flaws that were pointed out, I still can't imagine doing my work on windows vs linux. Even if I do have to sometimes spend effort battling linux's idiosynchracies. At least when something goes wrong in unix, I can probably go and find/fix what went wrong because the underlying system is mostly plaintext and transparent. Versus when something goes wrong in windows, it's more of a blackbox to me. But that could just be me not understanding windows enough.
NAME dd - convert and copy a file
Part of it is that it's way more annoying to maintain a little tool than to make it part of a bigger project with a much bigger community, it's also far harder to discover than just being in the docs a bigger project
And from the comments, it sounds like I'm not missing much.
Author, don't treat your readers like idiots. I can guarantee some of them are much smarter and more experienced than you.
"...great and perfect" is a strawman. Whether "some people" think that is irrelevant.
Some of this article is interesting, but the fact of the matter is 40-year-old systems have signs of being 40 years old. If "fixing" everything were easy, it'd be done. Tabs in Makefiles throw off the uninitiated for 10 minutes, then they learn, shrug and move on. These scars and stories are part of the package.
Reading further, some of this is just incorrect...
>That’s not to mention the fact that critical UNIX files (such as /etc/passwd) that are read upon every (!) call, say, ls -l, are plain text files. The system reads and parses these files again and again, after every single call!
Not on my system.
> It would be much better to use a binary format. Or a database.
On my system, it is ( running "ls -ld ."):
kamloops$ uname -a
NetBSD kamloops 7.99.64 NetBSD 7.99.64 (GENERIC) #26: Thu Mar 2 07:15:26 PST 2017 root@kamloops:/usr/src/sys/arch/amd64/compile/obj/GENERIC amd64
kamloops# dtrace -x nolibs -n ':syscall::open:entry /execname == "ls" / { printf("%s -%s", execname, copyinstr(arg0));}'
dtrace: description ':syscall::open:entry ' matched 1 probe
CPU ID FUNCTION:NAME
0 14 open:entry ls -/etc/ld.so.conf
0 14 open:entry ls -/lib/libutil.so.7
0 14 open:entry ls -/lib/libc.so.12
0 14 open:entry ls -.
0 14 open:entry ls -/etc/nsswitch.conf
0 14 open:entry ls -/lib/nss_compat.so.0
0 14 open:entry ls -/usr/lib/nss_compat.so.0
0 14 open:entry ls -/lib/nss_nis.so.0
0 14 open:entry ls -/usr/lib/nss_nis.so.0
0 14 open:entry ls -/lib/nss_files.so.0
0 14 open:entry ls -/usr/lib/nss_files.so.0
0 14 open:entry ls -/lib/nss_dns.so.0
0 14 open:entry ls -/usr/lib/nss_dns.so.0
0 14 open:entry ls -/etc/pwd.db
0 14 open:entry ls -/etc/group
0 14 open:entry ls -/etc/localtime
0 14 open:entry ls -/usr/share/zoneinfo/posixrules
kamloops# file /etc/pwd.db
/etc/pwd.db: Berkeley DB 1.85 (Hash, version 2, native byte-order)
...All the bluster (some of which is interesting), then at the end walks it back:
> So, I do not want to say that UNIX – is a bad system. I’m just drawing your attention to the fact that it has tons of drawbacks, just like other systems do. I also do not cancel the “UNIX philosophy”, just trying to say that it’s not an absolute.
Shame about the title... But maybe that's what landed it here on HN (?)
Edit: explain the "ls" command actually run.
Exactly, and this is the sort of thing that can be done with open source software. It may not even be a lot of code depending on how it is approached.
> [...]
> Anyway, the correct answer is: `ssh a@a "rm -- $(printf '%q\n' "$A")"`
In zsh it's just `ssh a@a "rm -- ${(q)A}"`.
The Unix philosophy is essentially the opposite of the Apple philosophy. It gives you flexibility and composability at the cost of simplicity and the overall experience.
The optimal solution tends to be somewhere in-between. If you look at Linux, it's actually a monolithic system (which goes against the Unix philosophy); the popularity of Linux is in itself proof that people do want a single cohesive product - If the Unix philosophy was the best approach, we'd all be using Minix by now.
I use Ubuntu (Gnome) these days. The only thing I miss from Windows is Windows Explorer. Nautilus just doesn't cut it in my opinion; I always end up browsing the file system with the command line. That said, I still prefer Nautilus over OSX's Finder.
A good example is ArcGIS which on the surface is ridiculously monolithic. But within the toolbox function are several hundred programs that do only one thing and are composable. This approach is also seen in video or image editing workflows where a user works with a particular set of tools. The main difference is that the programs use a types system that is appropriate to the domain rather than just text.
The OS only really exposes an interface for working with OS level objects. That sometimes aligns to a workflow but not always. And we should not expect disciplines to align their techniques to OS level objects if that is not a good fit for the actual domain.
It's entirely possible to have a microkernel with a Unix-like system; HURD attempts this. Microkernel vs. monolithic is an entirely separate issue.
No Mr. Askar Safin. YOU are lucky that I am reading it.
We're trying to use a server OS on a single-user machine, complete with all the management cruft that comes along with a server-based OS. Consequently, we don't bother re-thinking what the user needs vs. a sysop.
The Windows registry/powershell approach has the niceness of passing pieces of data instead of one big blob of text that has to be re-parsed at every step, but with the drawback of verbosity and fussy static typing.
Being able to directly pass s-expressions between between programs without the format/parse/typing hokey-pokey of Unix and Windows would be nice.
Also, ZetaLisp and Common Lisp have way more types than those supported by S-expressions. For example, they have real vectors/arrays, structures, and full objects.
Don't assume all of the Lisp world uses just the subset of Scheme used in the first couple chapters of SICP.
Genera did not have programs, it is all just function calls.
There was a multi-user LMI Lisp machine that ran Unix concurrently, that used streams to communicate between the two Lisp and one Unix processors: http://www.1000bit.it/ad/bro/lmi/LISP-MachinePR_Photo.pdf
Formats for inter-process communication are hard and far from a solved problem. Just look at what has come out in the past 10 years: Thrift, Protocol Buffers, Cap'n Proto, FlatBuffers, SBE, probably twice as many others I haven't heard of.
Next thing - bitch about how Earth works, with every region, country and even, gasp, city is different because a few people a long time ago decided "this is the way to go".
(Windows also has a similar problem.)
Who the hell 'develops' in shell? It's a glue language, not a development language. I've never heard anyone say "We're a shell shop".
anyway, is something like [this] result of development or gluing things together?
[this] https://github.com/roman-neuhauser/haveopt/blob/master/s/hav...
I read this, and was like "screw that, I'm not going to read it," and then was like, well let's see, and after reading, wish I had stuck with my initial gut instinct to close that tab. smh.
My GPU has 4 times as many transistors as my CPU, and for parallel tasks, it computes stuff 50 times faster. Just too much complexity for a file, even with ioctl.
I think that ideology is the main reason for the current state of 3D graphics in nix and bsd based platforms.
However, in my comment I didn’t meant text files. I meant that in *nix, a GPU itself is a single file, /dev/dri/cardN. All that GPU’s complexity is squeezed into a single ioctl system call for that special file. The approach is IMO one of the reason why Linux still doesn’t have reliable 3D acceleration support.
Here’s a long article why Linux ain’t ready for desktops, note the author listed 3D acceleration on the top:
https://itvision.altervista.org/why.linux.is.not.ready.for.t...
i'm still to see any complainer grown an adecuate amount of facial hair, though. (incompetently) reinventing some bad idea from before you were born does not cut it, you know?