All these subsystems have one thing in common: they are incredibly complex and very hard to get right, but seem easy on the outside.
These are complex to make robust, but they are not hard concepts. I think everyone should write their own GC, write a filesystem, and handle their own memory layer, at least once.
Treating lower levels of the stack as "too complex, there be dragons, just write some Javascript that hits RESTful API for JSON" is a great way to ensure you never progress as a software dev or an engineer.
I agree that reinventing the wheel poorly - without understanding geometry or physics - is a terrible idea. But I also think that we don't have enough people nowadays making toy wheels to learn about geometry and physics.
For each of the things you mentioned, there are concrete and good reasons why they grow complexity: filesystems need to be robust to media corruption and power faults; memory allocators need to be highly tuned to OS and architecture memory layouts; GCs need to be optimized for the code patterns of the particular language they serve, as well as understand the same OS and hardware memory constraints as allocators.
But much of the complexity of these layers is not intrinsic to the problem, and not related to the above. In fact, most of the complexity and bugs of production code at these layers have to do with legacy compatibility, or code inheritance. (If you look at what Theo says about this specific OpenSSL bug, it arises because the OpenSSL team wrote their own allocator that doesn't use malloc() protection bits because those are not compatible across some platforms.)
As the old saying goes: in science, we advance by standing on each others' shoulders; in software, we stand on each others' toes.
[NB Ken Shirriff's blog articles on Bitcoin are a superb example of someone coding in an effort to understand a system - but I don't think he is attempting to write a real Bitcoin library]
Of course. But not in production. Which is a detail missed by the snarky one-liner, maybe because one-liners suck. Still, the point isn't that people shouldn't ever write them, just that they shouldn't actually use the ones they developed while not being a part of a team of experts specializing in the issue at hand.
Yes, yes, I didn't quantify my statement with "for production use".
Your sentence I quoted above was exactly my point: they aren't necessarily hard (or at least they don't seem hard), but they are very, very difficult to get right (robust). Getting systems like these to work well in practical production use is more than 80% of the effort.
I wasn't saying that no one should learn how a garbage collector works by implementing one. My point was that no one should implement their own garbage collector for a production system unless they already became an expert in the field of garbage collectors by implementing them for the last 10 years or so. Same goes for memory allocators. If someone thinks these are simple systems, it means they don't know what they don't know.
After writing your own allocator, you'll never view malloc() as a cheap and simple operation again!
Hopefully the act of writing your own malloc implementation will be enough to convince you not to use it in anger, but it's probably best to be clear on the subject.
I mean, there is an extremely small chance that anybody will adopt your library if it is unquestionably better than the dominant alternative. We don't actually need to discourage people, we need to do everything to encourage them. Sure, 99 of 100 will be no better than our present crap. What does it matter? Few will adopt it anyway. The tragedy is that the better alternatives will probably meet the same fate.
It can also mean that existing implementations are poorly documented and research papers on the subject place no importance on implementation details. Of course you should not use your first implementation in production, but that is as obvious as not letting someone do brain surgery who has no prior training in it.
The cost in effort is highly non-linear. It may be fairly straightforward to get the 80% solution. I agree that it's a problem when devs bring their home-grown 80% solutions to the table when you need a 99% solution.
I make games. No one dies if there's a bug in the AI. No one may even notice.
And have teams of operating systems engineers whose entire existence is predicated on them, rather than them just being an afterthought so you can get some other stuff done. If the OS does it, you shouldn't.