Comment by ThePhysicist | Hacker News Reader

Comment by ThePhysicist | Hacker News Reader

I absolutely love writing programs in Go and at my current startup we're basing all our core backend infrastructure on it.

One thing that I find extremely troubling about Go's dependency handling though (and which at least partly contributes to people's frustration I think) is the mixing of the concepts of a namespace and a resource identifier in Go's package names. It's absolutely reasonable that package names should be unique, making the package name double as a storage location is not always a good idea though (IMHO): For example, we recently switched from using gitlab.com to our own private Gitlab instance, which runs under a custom (non-publicly resolvable) domain. Since all our packages were named like "gitlab.com/.../..." (to make them "go get"-table) we would have to rename all of them to make them "go get"-table from the new location. In the end we solved it by defining a git URL rewriting rule that rewrites the old to the new URL when fetching the repository. This ensured that we could work primarily on our new Gitlab server while maintaining the possiblity to build packages on the gitlab.com server. While that works I think it would have been a better idea to keep package names and storage locations separate, or at least give users a way to override/modify the resolution of packages into locations using a more standardized approach: Such a mechanism could then allow for different name resolution strategies (like a package file, a list of standard repositories) while falling back to the standard way of resolving packages into URLs for "go get". Granted, this would add another layer of complexity, but as we already need a separate file for proper vendoring (to e.g. specify versions) it wouldn't hurt too much to also put (optional) information about storage locations into that file.

Just my 2 cents, apart from this minor inconvenience I'm absolutely happy with Go and its ecosystem!

if you want to decouple the namespace from the repository location you can!

https://golang.org/cmd/go/#hdr-Remote_import_paths (read until the end)

or TL;DR: https://blog.bramp.net/post/2017/10/02/vanity-go-import-path...

You still can't really.

If I have a canonical import path of "mydomain.io/foo", and I host html there pointing to github or whatever, I now have a different single point of failure: the domain. If I lose control of the domain or the

Most package management systems have one additional layer of resolution: a centralized pluggable repository location which can be configured to some default, and then lookups are made against that instead of against a url specified in the source code.

For example, with cargo, I can open my ".cargo/config" file and specify "registry.index = whatever-url.com", and then for package "foo" it'll look it up on that registry.

If I forget to renew the domain, I don't have to change a bunch of import comments in my source code to update the registry location; I have to update one cargo config file.

I believe the solution you're looking for is Athens[0], right?

[0] https://github.com/gomods/athens

> If I have a canonical import path of "mydomain.io/foo", and I host html there pointing to github or whatever, I now have a different single point of failure: the domain. If I lose control of the domain or the

> Most package management systems have one additional layer of resolution: a centralized pluggable repository location which can be configured to some default, and then lookups are made against that instead of against a url specified in the source code.

TBH, these two seems to contradict each other. You don't want to use your own domain, because that's a SPOF that you could lose control over. Whereas the centralized repository is a SPOF that you never had any control over to begin with. Seems strictly worse to me. And the nice thing about the chosen approach is that it can coexist with what you seem to be wanting: gopkg.in exists and is exactly that. A centralized, hosted (and free!) registry for go packages with its own registration and authorization process.

What I find so beautiful about the go-get solution is a) that you can build any other mechanism I know of on top of it - i.e. the user can decide on their own tradeoffs of convenience/scalability/security/future-proofedness… and b) that domain names already provide an elaborate, globally agreed upon namespace that can separate ownership from administration and hosting, with its own takedown-semantics… It's decentralized, existing, robust infrastructure. It gives full control to the user. And it solves a whole bunch of technical challenges already, for free. It means the Go team doesn't have to manage user accounts of package maintainers and it doesn't have to have policies of what code to accept or not accept - as it can't enforce them.

It's really quite brilliant, IMO.

The point is that I don't have to change any of my source code in order to change what registry I point at.

I can use "foo.registry.com" with no changes to my rust source code.

If I want to switch go from "old.com/x/package" to "foo.com/x/package" I have to rewrite all the source code first.

That's the weird coupling which makes go not so configurable; code has no place dictating where it's downloaded from, yet go packages do that. I can't use the go package management ecosystem if my canonical import path doesn't resolve (e.g. dep won't work, go get won't work, vgo won't work).

I never have to worry about my rust code having a magic comment in it which makes the package manager incapable of fetching it from some arbitrary differently named mirror. That's simply not possible in a well designed package management system.

In our CI setup we ended up fixing it like this (which I still consider very hacky though):

    git config --global url.https://our-new-domain.com/.insteadOf https://gitlab.com/

This allowed us to keep our code unchanged and still be able to build it on different CI servers (e.g. our public as well as private instances). Specifically, for Gitlab our replacement config looks like this:

    git config --global url.https://gitlab-ci-token:${CI_JOB_TOKEN}@${CI_HOSTNAME}/.insteadOf https://gitlab.com/

This assumes that all packages hosted on gitlab.com are available on our local Gitlab instance, which can be problematic as well (for example if we would import/use an open-source package hosted on gitlab.com along with our own code). In that case we would need a replacement config that would only match our groups/namespaces on gitlab.com.

I vastly prefer the Rust way of doing things though, as I don't think using domains as a namespace is a good way. For example, most code hosting sites will shut down or be deprecated eventually (possible even Github at some point), which for Go will lead to a lot of headaches since there's no way to know where a given package will end up when it's no longer available at its original location.

> The point is that I don't have to change any of my source code in order to change what registry I point at.

Apparently it's a matter of opinion, but I consider that a feature, not a bug. It means the import path, as the identifier of the package, identifies the package. If two source files refer to the same name, they mean the same code.

> code has no place dictating where it's downloaded from, yet go packages do that.

That is still not actually correct. The webserver listening on the domain is dictating where the code is downloaded from. i.e. the owner of the identifier of the code you are downloading.

The basic assumption is, that if you take some code and put it somewhere else, you are assuming ownership, so you should actually own the identifier of that code. If someone wants Library X, they should be sure to actually get Library X, not your fork Y.

> I can't use the go package management ecosystem if my canonical import path doesn't resolve (e.g. dep won't work, go get won't work, vgo won't work).

To me, this seems to be a weird and unfair comparison.

gopkg.in exists. There is nothing in the Go ecosystem that is preventing what you want to have. The equivalent to Rust's cargo+crates.io isn't go-get+custom-domain, it's go-get+registry. And the equivalent to the import path "github.com/user/package" isn't (import of the "package" crate plus a mirror-directive in your cargo config), it's just the string "package". Because that's the identifier of the crate.

You could make the argument that Go still doesn't allow you to have mirrors, which is fair on the one hand, but will be solved with modules on the other.

Really, IMO it's just a fallacy to think of the import path as a download location. It's an identifier and the only reason the domain must resolve, is because go-get must consult the owner of that identifier what code to fetch. And DNS is the canonical tool to do that. Just like you need to resolve a hostname to get a Let's Encrypt certificate or use the Google Webmaster tools - you have to prove ownership of the name.

> While that works I think it would have been a better idea to keep package names and storage locations separate

Then why didn't you? Go had that ability since the inception of go-get. You could have used your own domain to name your packages from the beginning and have avoided that.

I agree it sounds a bit “painful”.

But I think on the plus side it was very easy for you to reason about the tooling to get your simple solution going.

Thinking more of: It is great that you can kind of apply “Dependency inversion of the tooling” without jumping to too many hoops to solve your problem.

Ah, it's a variation on the URL vs. URI vs. URN debate again.

Reply on news.ycombinator.com