Comment by db48x | Hacker News Reader

Comment by db48x | Hacker News Reader

Parent Full thread

I’m not sure that’s a great example. mmap(2) is a swiss army knife of a function. There are many crates that build all kinds of things on top of mmap, and they’re not all interchangeable. Allocators, file io, gpio, actual virtual memory mappings.

My view is that when you develop a product, you have to own everything. The users don’t care if the bug comes from code you wrote, or code in a library, or the language’s standard library, or the OS. You have to fix the bug no matter what caused it. It doesn’t matter if the code came from a vendor or a language designer or stack overflow; you are the one responsible for fixing it if something goes wrong. Non est salvatori salvator, etc.

With that perspective, I don’t think that 23 crates is terrifying. I’m going to look them all over and either pick one, or write the 24th crate myself. The result is the same either way.

I do agree with you about APIs though. It is nice to find a crate where the author has had the confidence to stabilize their API and declare the version to be 1.x instead of 0.x. But at the same time I recognize that getting to that point requires some real software to use the crate, to create the feedback loop. If nobody ever used a 0.x crate, no crate would ever get that feedback.

>My view is that when you develop a product, you have to own everything. The users don’t care if the bug comes from code you wrote, or code in a library, or the language’s standard library, or the OS. You have to fix the bug no matter what caused it. It doesn’t matter if the code came from a vendor or a language designer or stack overflow; you are the one responsible for fixing it if something goes wrong.

So then using a programing language that has a big standard library and a more rich and stable ecosystem is a big advantage. If I want to do a network request, parse a json file, parse or format a date or some other trivial thing I prefer to have a good enough built in way to do it then evaluate 15 packages or write it myself. Otherwise you get in a shitty situation where you inherit some projects and it has 20 dependencies with vulnerabilities, 20 dependencies abandoned, some dependencies that are incompatible with some new version of the language/ecosystem causing issue if you would like to upgrade.

Sometimes. Using someone else’s code is only a benefit if it saves time overall, compared to writing it yourself. You have to count both the time to write it and the time to maintain it over however long your product is active. You have to assume that you’ll be maintaining it yourself either way.

Also, don’t forget that languages with big standard libraries, like Python or C# or Java, always end up with oodles of stuff in there that is deprecated or incompatible. That’s as bad a sign as anything.

But I do agree with you, a lot of the time you have inherited the mess rather than writing it yourself. Not much you can do about it, except to clean it up and then dodge better next time. Going back to Rust specifically, Cargo gives you some tools that make the mess easier to clean. The Rust language also helps, because you can safely use multiple versions of your dependencies, when it turns out to be necessary.

>Also, don’t forget that languages with big standard libraries, like Python or C# or Java, always end up with oodles of stuff in there that is deprecated or incompatible. That’s as bad a sign as anything.

This does not happen that much, and in Java and .Net world if some functionality is deprecated there is a replacement, I think I only remember some possible unsafe or ineeficient functions were deprecated so you use the better ones. I don't have experience with Rust , only with node/npm and is a hell , for some reason is decided to split stuff in super small pakcagtes of various quality, today I sepnd hours debugging a npm freeze caused by some package , in the end the cause is probably a shit npm/node implementation that crashes when some specific git version is installed , but debugging this I discovered that the unit testing packages the project uses(I inherited them) instead of beeing one or few packages are a few, and for some reason one of them had a weird dependency that it should not have(a dev only despondency ) and this dependency was also just pulling directly from GitHub ...shit in a few years when GitHub is gone lots of things will stop working (I also had issues with scripts because someone changed master into main because American politics...).

I personally prefer the Java or .Net model, a big standard lbirary and then for most important things you have a small number of options since the community did not wanted to do CV driven development, and most of the time you find all you need in a single library with no or few dependencies. But Rust does not have the Java or .Net money to hire devs to work on the boring stuff of correctly implementing standards like json,XML, date&time and keep maintaining it - (not sure how Python did it) and Rust community seems to be inspired byt node community a lot and this is bad.

> With that perspective, I don’t think that 23 crates is terrifying. I’m going to look them all over and either pick one, or write the 24th crate myself. The result is the same either way.

In a fine-grained ecosystem, the problem is transitive. You aren't just picking one from 20+ for each of your direct dependencies, you are also trusting that they in turn did just as much diligence as you did in picking their direct dependencies and so on. Curation and pruning would at least help mitigate the scope of the problem.

> ... or write the 24th crate myself.

This is a good example of the xkcd joke about standards proliferation. If I found myself in this situation, I'd prefer to keep the 24th private to my own package. DRY has its limits. Sometimes you've just got to specialize.

I'm aware of the irony in saying this in the context of mmap. But in this case, a thin wrapper that reflects the system call's semantics ought to be available in a common `posix` crate that others may freely rely upon to build their higher-level services. There's no good reason for e.g. ripgrep to have to pick and choose among a dozen flowers for it.

> In a fine-grained ecosystem, the problem is transitive.

Absolutely. You definitely have to own the whole stack. Ultimately, you may have to fix bugs at every level, from your own code all the way down to the OS. Big tech companies do this explicitly, with kernel development teams. Small companies do this by occasionally upgrading to the latest version of Ubuntu and hoping for the best.

> But in this case, a thin wrapper that reflects the system call's semantics ought to be available in a common `posix` crate that others may freely rely upon to build their higher-level services.

It is available. If you want to directly call any function in the C standard library, use the libc crate. Here’s the documentation: https://docs.rs/libc/0.2.109/libc/fn.mmap.html

All the other crates on crates.io that you find when you search for “mmap” are higher level abstractions over the raw syscall. They have specific purposes, like file io, or creating circular buffers.

Reply on news.ycombinator.com