Hacker News Reader

Comment by jnwatson | Hacker News Reader

"But they won’t be happy about it and it may bring some negative vibes to your tropical vacation, financed by the millions you’ve made from maintaining a FOSS project."

I love this line.

There's a tragedy-of-the-commons problem with respect to version pinning. What is good for you isn't good for the community.

If your project is closed source depending on open source modules, go for it. However, if your project is open source, you're caught in the middle between the interests of your sanity and your downstream.

If you pin too tightly, it increases the likelihood that your downstream users won't be able to construct a valid set of versions. If you pin too loose, then you're rolling the dice that the next upstream update won't break something.

The only real solution is to constantly evaluate your upstream and constantly update your pins, which is quite a treadmill.

View on HN

This really is a solved problem within Rust & Cargo. That's why Cargo is such a critical part of Rust & why it really gains traction. The entire build system is so sane that I'll gladly pay the penalty of having to read & write Rust (as a novice) to have a sane build system I can trust to keep many parts of my build supply chain safe.

As an example, you're not stuck. If A pins version <= X of dependency B, but C depends on B > X, component D can include both A & C as dependencies & it will still build & work correctly. https://stephencoakley.com/2019/04/24/how-rust-solved-depend...

There is one thing in Rust that's not so rosy but I think could be solved if it's a big enough problem in practice. If a shared dependency has a version that satisfies all preconditions across all downstream libraries, that one dependency will be used. Where that can fall down is if that version actually does break one of your dependencies because of some oversight on the community of suppliers. A nice addition would be a way for downstream to add additional constraints to override things (like blacklisting version dependency chains). That might become more of an issue if there's a package that gets popular enough & then goes stale (i.e. dealing with garbage in the broader Rust community).

> If A pins version <= X of dependency B, but C depends on B > X, component D can include both A & C as dependencies & it will still build & work correctly.

Actually, I believe that's only true if the two versions of Bs have semver-incompatible version numbers.

So, for instance, if your dependency A requests B=1.5 and C requests B=2.3, then cargo will do what you say and compile both versions separately.

But if A requests B=1.3 and C requests B=1.4, cargo will try to find the highest semver-compatible version that satisfies both constraints. If A pins B<=1.3 and C pins B>1.4, cargo will give you an error.

https://doc.rust-lang.org/cargo/reference/resolver.html

IIRC, npm already does the same thing. If A and C depends on two compatible version of B, it will make them use the same package. If two version aren't compatible, it will use two package and it still works.

Version pinning does not compose.

We keep treating things like lock files as if there is only one kind of use for code and it's just madness.

Should end-user applications use pinning? Yes, and aggressively. Should library writers use pinning? Please, no.

I think part of what makes these discussions difficult is that each system uses similar-but-different approaches. For example, in Rust, even if your library has checked in Cargo.lock, it won't be used for resolution of that library. This suggestion of yours is literally coded into Cargo directly. But someone coming from Node may not instantly get that, even if both ecosystems "use semver."

Notwithstanding this (that Cargo.lock doesn’t apply to libraries), you can pin in Cargo.toml by specifying exact dependencies: `foo = "= 1.2.3"` will accept only version 1.2.3.

Yeah, this is a good point. I don't know that I've ever noticed it in the wild, but it is possible.

I can’t upvote enough.

If you’re writing a library or a plug-in that is going to work with different major versions of some containing application, you have to be somewhat flexible in what dependencies you require. You’re going to be coexisting with other plugins and libraries. If you want requests 2.23.0.1, and the other one wants 2.22, and the other one wants 2.24, all to simply do requests.get, madness follows from the people trying to use the library.

This leads to either: forking, serial pip installs, or madness. Or all three.

Libraries shouldn’t have a preference for tzinfo, or certifi. They certainly shouldn’t have conflicting preferences for them, and the only way that can happen is if the libraries defer to the application for the version of widely used core dependencies.

There are exceptions, and generally fall along the lines of libraries that have tightly coupled dependencies, or cases where the specific advanced features are used. I’m thinking here of awscli and boto, but there have been epic disagreements on both between awscli and awsebcli.

This is really the meat of the issue I think. The root of the problem AFAICT is that there are people involved in these software systems. People who can make mistakes, people with emotions, egos and scarce time.

I don’t see how any of the author’s very valid points about unintended consequences go away if, instead of pinning to 1.2.3, you pin to 123. Or 2021-03-02, or 57d4fca9.

Software is only becoming more modular and reusable. We need more discipline, not less by throwing out protocols while blaming them for not solving the problems we create with them ourselves. That’s not “taking responsibility” IMO.

Reply on news.ycombinator.com