Makes me appreciate what Go eventually did for their module system with respect to the issues raised here for Rust.
What does go do when you have different deps depending on different versions of the same common dep?
https://doc.rust-lang.org/cargo/faq.html#why-do-binaries-hav...
> If a library ends up being used transitively by several dependencies, it’s likely that just a single copy of the library is desired (based on semver compatibility). If Cargo used all of the dependencies' Cargo.lock files, then multiple copies of the library could be used, and perhaps even a version conflict.
> In other words, libraries specify SemVer requirements for their dependencies but cannot see the full picture. Only end products like binaries have a full picture to decide what versions of dependencies should be used.
tl;dr, Go does a BFS of the dep tree, selecting the maximum version found. There are no ranges, so you can only use a version which has been listed.
"minimum" comes for a minimal implementation which can create reproducible results without a lockfile
> it adds a line to a `go.sum` while with a hash of the code at the version specified
Cargo.lock also contains a checksum
> You can distribute your code without a copy of the dependency
Also true in rust, and the default way of using rust/cargo.
> If the hashes are different, an error is thrown.
Also true in rust.
For an example of what this looks like: https://github.com/servo/webrender/blob/54b725be37f13b166946...
You haven't described anything different between go and rust in your comment since every feature you've pointed out applies equally to both.
Versions are git tags or commits. Dependencies are specified in a go.mod file, specified here: https://golang.org/ref/mod#go-mod-file
From the following go import path:
import "mywebsite.com/foo/bar/baz/v2/fizz"
1. mywebsite.com has a 'meta name="go-import"' html tag in it when accessed by the go tool that points to a git repo/svn repo/etc, and the contained package has the folder structure "foo/bar/baz/v2/fizz"
2. mywebsite.com/foo has a 'meta name="go-import"' html tag...
(several obvious cases omitted)
3. mywebsite.com/foo/bar/baz is a git repo that has no version tag, but has a "v2" folder in it
4. mywebsite.com/foo/bar/baz is a git repo that has a version tag with v2.x.y
5. the go.mod has a replace or other directive for this import path
The 4th case is the relevant one here. github.com/foo/bar/v2 means that the bar repo has a v2 tag usually.
So, what version of that repo is used if there's v2.0.0 and v2.1.0? Well, the version is the smallest version that you or any module you depend on uses, so you simply have to recursively parse all go.mod files to understand the version used. Much simpler than Cargo.lock.
How do you enable or disable features in a sub-go-module? Let's say I have a module that has a "+ui" or "-ui" build tag because you can optionally build an openGL ui or such.
In rust, this would be a 'features = [ "ui" ]' in your Cargo.toml. In go, you would change your own compilation step to be "go build -tags ui"... And if you have two dependencies with those flags, and you wish to make them opposite values, that is simply impossible in go even though it's trivial in rust.
There are other differences, but I feel like I've already made my point. My main point is that go modules make import paths way more opaque. It used to be that every path element was either part of the import path or a folder on the filesystem. Now, it's either part of the import path, or a git tag, or a folder, and none of those things are obvious.
My secondary point is that go has a much simpler set of requirements than rust, mostly due to missing important features rust has.
There is no NPM or PyPi like host, sources originate from code hosts
Modules are prefixed with a domain name (avoid dep confusion)
Hashes are calculated on the module, it will fail if it is not correct
GoProxy has a global transparency log (like a blockchain, but not) so that everyone is contributing and sharing hashes
*there are special versions (v0.0.0 and branch names). They have special meaning, and get fixed with the access date and commit hash
For the curious: https://golang.org/ref/mod and https://go.googlesource.com/proposal/+/master/design/25530-s...
>Versions are only importable from tags (semver style)
Tags are mutable in git, which sometimes causes problems. Not that I think they had much choice here - git and tags are the de-facto community standard, they're just using it like everyone else.
>There is no NPM or PyPi like host, sources originate from code hosts
>Modules are prefixed with a domain name (avoid dep confusion)
Agreed, I like this quite a bit... but it comes with the significant risk of domain name or hosting changes. Github probably won't add the metadata to redirect your old module name to your new non-github name, for instance. I've watched several modules fight through this sequence, and the end result is always "can't do anything, contact your users yourself / hope they read the readme".
You win some, you lose some. In large part I think this part of Go modules is better than others'.
>Hashes are calculated on the module, it will fail if it is not correct
Nearly all lock-file-based dependency systems do this, Cargo included.
>GoProxy has a global transparency log (like a blockchain, but not) so that everyone is contributing and sharing hashes
Publishing a log now doesn't make it any safer tomorrow, it's purely about convincing you to trust them. This is precisely the same attack target as an NPM or PyPi. The same party hosts the tar'd source files + documentation + checksums: if it ever gets changed there, it's changed for everyone. Lock and go.sum files will only protect you from changes to your previously-known dependencies.
They've done a pretty good job making goproxy/gosum easy to run yourself, and you can skip past the goproxy and download directly from the module's owner (which is a domain name, which works great here), which mitigates this specific risk at least. But no, they're absolutely running a pypi, they just don't let repo owners make changes directly (only by committing changes, e.g. module revocation).
> if it ever gets changed there, it's changed for everyone
sum.golang.org has a hash chain, so you have to change everything that comes after a malicious hash change. So it does seem safer the more time that passes.
Go uses zip files, as a fyi.
I've had more trouble with custom domain based Go mods than the major code hosts. I've come to avoid those on custom domains as much as possible.
Is there a work-factor attached to these? I kinda don't think so, the only reference I can see is "sha256" (I have not checked the code.... nor is the public proxy the same thing they run, though it should be good enough for this purpose). Unless it's "costly", the time needed to recompute the whole history of every module is probably well below one second. The chain just makes it so you don't need to do that on every single update.
It's a good structure for making attacks more noticeable (you can't change any piece of history without everyone who already has that history noticing), and it enables efficient and trustworthy partial updates (backfills? meh), so I think it's a very solid addition and I'm glad it exists. But it really doesn't protect against much in the way of practical attacks - it's a canary, not a preventative measure, mostly serving to say "see? we've been trustworthy, you can trust us going forward".
(edit: I'll completely believe there's a small niche of attacks that it prevents... but I can't think of any that aren't massively overshadowed by other also-possible attacks in the same situation)
>I've had more trouble with custom domain based Go mods than the major code hosts. I've come to avoid those on custom domains as much as possible.
Yeah, custom domain ones have definitely had more problems - the proxy helps a fair bit there :) I don't mean to imply I'm unhappy they're running a PyPi-alike; I'm absolutely thrilled that they finally admitted they need it, and it's a decent implementation too. But that's pretty much all it is.
That is true. The global transparency log is just to prevent a package version change after it has been recorded in the log.
This is not true. Only Google sum server contributes to the hashes.
really sucks
> There is no NPM or PyPi like host, sources originate from code hosts
Bad idea. Easy for packages to disapear.
> Modules are prefixed with a domain name (avoid dep confusion)
This seems like more trouble then its worth most of the time