Hacker News Reader

Comment by mseepgood | Hacker News Reader

> For all practical purposes dependency management for programming languages is a solved problem

It's obviously not solved satisfactory.

Yes, it is. Everything can be improved, but I see very few complaints with Cargo, and, most importantly, not complaints that would be solved with minimal version selection. (Most complaints I see are about the lack of package namespacing.)

One of the ways I'm hopeful that cargo can improve in this area is through the work being done on rust-semverver. If crates.io can either determine semver compliance or, perhaps, deny releases that are not properly semantically versioned, it would give cargo more license to select a single version to satisfy diamond dependency graphs. Those diamond dependency graphs are one of the ways that package management is famously very much unsolved.

I agree that Cargo is awesome and I would be happy to see Go follow suit, but I interpret "package management is solved" to mean "the industry has standardized on a single scheme". There does seem to be convergence in this space, but it's a relatively new phenomenon.

Yes, the industry has absolutely standardized on a single scheme: manifest plus lock as separate declarations, SAT solver, single dependency version per compilation unit.

That may be what things are converging on but in my opinion, any system that imposes a "single dependency version per compilation unit" policy is not a "real" dependency management solution. That approach creates a tremendous amount of friction because it pretty much ensures dependency hell if there are transitive dependencies with separate release cycles (and there very much should be able to be).

Edit: I just realized that we may have a different definition of "compilation unit". If you mean "single program" then my point stands. However, if you just meant some sectioned-off part of the program, then I think the drawbacks of that are far less. Basically, I think that it is useful to be able to have access to multiple versions of a datatype (e.g. if you want to write a converter from old -> new) but that use case is far more fringe.

As a sysadmin I object to the notion that old to new conversions are a "fringe" use-case. Pretty much all my coding (I'm a "devops" sysadmin-type) is exclusively "I need to wrangle all versions of thing 1 into a type of thing 2 or between several versions of thing 1".

If backwards compatibility is important, then letting me use multiple versions side-by-side in the same modules easily is damn important.

To be clear, Cargo fits the latter definition; you can't depend directly on two different versions of a package, but you can transitively.

If that's the case, it's a pretty recent development. Python and JS didn't have this until very recently. Yeah, Go is a bit late to the party, but hardly worth the dramatic criticism in this thread.

No, it's hardly a standard. It's just a common approach. The article presents another approach which differs on the three points you mention: manifest plus lock as separate declarations, SAT solver, single dependency version per compilation unit. And I think what Russ proposes is a really interesting tradeoff in the design space.

It is the _defacto_ standard and has been for many years. It is what's expected by the absolute and vast majority of software developers.

> I see very few complaints

If you live in a filter bubble (survivorship bias).

Only because the golang core team seem to insist on reinventing every wheel either without considering prior art, or at least without considering why prior art went with a particular approach.

I swear I should put together a list of these instances.

> Only because the golang core team seem to insist on reinventing every wheel either without considering prior art, or at least without considering why prior art went with a particular approach.

The popular criticism is that Go isn't reinventing enough; that it's stuck in the 70's and it's not sufficiently innovative.

I think it's two sides of the same coin.

I've heard go described as a language that doesn't try to solve any hard problems. Let's set aside for a moment whether or not that's a positive or a negative. The bigger problem is that they also seem to completely ignore everyone else's existing solutions to hard problems too, which leads to the criticism that it's stuck in the 70's.

Sure, a lot of the existing solutions to some of these problems aren't perfect or involve tradeoffs that don't work for everyone. In general though, more recent solutions have built upon the lessons of previous approaches and now we've got languages and tooling that get a hell of a lot right, straight out of the box (I'm looking at you Rust / Cargo).

But go isn't doing this. They appear to be trying to come up with de novo solutions to problems from the perspective of the 70's. Whereas everyone else is "standing on the shoulders of giants", they insist on discarding the lessons we've learned over the past fifty years and running one by one into all of the same problems that led us to the solutions we have today. As an observer from the sidelines it's painful to watch them relearn these same lessons the hard way.

I largely agree, and still for everything that Rust and company get right and for everything that Go gets wrong and for how much optimism I approach Rust and company with, I'm still far more productive in Go than any other language (even the ones with which I have much more experience). The only way I can explain this is that programming language design is something of an art form, and the integration of features and tools is far more important than the set of features and tools.

Rust current package manager is third attempt which worked. Async saga also is approaching 3rd attempt before we have it working. For module system there was big backlash on simplifying it for new users. so effect will be more likely that is understood by few developers who have spent significant time.

That's a really good perspective. Thanks for sharing.

I think that's an inaccurate/misleading framing: the typical criticism is that it has seemed to ignore innovations from elsewhere, there's no suggestion that it needs to reinvent them. For instance, people consider generics a solved problem as there's numerous schemes invented by other languages, and critics think one of them should be able to work for Go.

Yes, but Go has specificities that make difficult to directly use the scheme used by other languages: Go is AOT compiled (we can't use C# generics), Go favors compilation speed (we can't use C++ template), Go lets the developer control memory layout with arrays and structs (we can't use OCaml generics), etc. Of course, Go should get inspiration from other languages, but adding generics is not easy as simply transposing another language implementation. There are multiple detailed proposals in the bug tracker that have failed for multiple reasons.

There are also CLU, Ada, Modula-2, Modula-3, Eiffel, Sather, BETA as possible inspirations for implementing generics, all the way back to 1976 (CLU).

As for C#, people keep forgetting .NET always had a JIT/AOT compiler since version 1.0.

Just because not everyone bothers to sign their applications and call NGEN at installation time, it doesn't mean it isn't there.

Also Mono always supported AOT compilation, it is the way Xamarin works on iOS and the deployment story on Windows Store since it was introduced on Windows 8.

I remember Mono with AOT compilation had some limitations regarding generics in the past, but maybe this has been solved since then.

Yes Mono has some limitations, but it isn't 100% like the iOS Xamarin compiler.

http://www.mono-project.com/docs/advanced/aot/

In any case, a limited form of generics is better than not having any at all.

They don't need to provide a turing complete implementation of generics, even a basic one like CLU had would already be an improvement.

Using go generate feels like the old days, writing generic code in C and C++ with pre-processor tricks.

I agree. I hope the Go team will tackle this issue, when they are done with dependency management.

Swift has essentially all the same constraints, and fretting about compilation speed with the C++ approach are hard to stomach given one of the widely-used replacement is equivalent: manually copy-pasting code, instead of letting the compiler do it.

In any case, the truth of the complaint about generics isn't actually relevant here: the rebuttal is to "people complain about Go not reinventing enough", and they don't, as they (truthfully or otherwise) think it is ignoring already researched and widely-implemented solutions to the problem and complain about this (that is, the complaint is "uninventing", not "not enough reinventing").

I didn't mention Swift and I should have. I agree it's close enough to Go to provide useful inspiration.

But even Swift implementation of generics has its issues and tradeoffs. Look for example at this thread about "Compile-time generic specialization" in Swift: https://lists.swift.org/pipermail/swift-evolution/Week-of-Mo...

Repeating that Go ignores "already researched and widely-implemented solutions" is getting old, really. Would you say that OCaml doesn't have shared memory multicore parallelism because it ignores already researched and widely-implemented solutions??? Would you say that Haskell garbage collector causes long runtime pauses because it ignores already researched and widely-implemented solutions??? The truth is that solving that kind of things is a lot more complex than just copy-paste the code from another language.

I'm... intimately familiar with generics in Swift. :) Like all systems it has trade-offs, but the decision it makes line up with the decisions you said Go wanted to also make.

In any case, that specific concern doesn't apply to Go, since it doesn't have overloading.

> Repeating that Go ignores "already researched and widely-implemented solutions" is getting old, really

Sure, but that still isn't the point of this discussion: it's not what people should be doing, or the underlying truth, it's what people are doing in practice and they are complaining about the Go team seemingly ignoring the past 40 years of programming language development, not complaining that they're not inventive enough.

You're right about the issue I mentioned which is irrelevant since there is no overloading in Go. Thanks for pointing it out.

I didn't know you worked on Rust and now on Swift :-)

What strategy does Swift use to compile a generic function or method? Does it generate only one polymorphic version of the code, or multiple specialized versions for each expected parameter types (often called code specialization or monomorphization)?

One polymorphic version, like OCaml or Haskell, not specializing like C++ or Rust.

The compiler can and does specialize functions as an optimization, but that's not necessary nor part of the semantic model.

I always thought that generating one polymorphic version would be a better fit for Go than generating multiple specialized versions.

It's interesting that Swift adopted this approach and confirms what you said earlier about Swift having essentially the same constraints as Go and being a good source of inspiration of Go.

What's the status of Swift on Linux, to write web services and connect to databases?

There's lots of testing for Swift on Linux (e.g. every pull request runs tests on macOS and Linux), and Apple recently announced https://github.com/apple/swift-nio which seems to have been very well received, e.g. the Vapor framework is completely switching to it for Vapor 3.

Thanks for the link. Does Swift provide something similar to goroutines?

>OCaml doesn't have shared memory multicore parallelism because it ignores already researched and widely-implemented solutions?

OCaml people definitely not ignore them, they are carefully investigating existing code and elaborating modern and efficient solution [1](unlike Go, which is too opinionated).

[1] https://github.com/ocamllabs/ocaml-multicore/wiki/Memory-mod...

I'm aware of the ongoing work to enable multicore parallelism in OCaml. It's the reason why I mentioned OCaml in my comment. My comment was rhetoric. My apologies if it was not clear.

That said, I don't see the link between the work done on OCaml multicore parallelism and Go "being too opinionated"...

If I follow you, Go people, unlike OCaml people, are "too opinionated", and don't "carefully investigate existing code" and don't "elaborate modern and efficient code". Do you realize how arrogant (and a bit ridiculous) this sounds?

Is Swift scheme same as Rust which is same as Java which is similar to C++? If not then it seems to me language developers decided to use an approach what suited their purpose.

No, they're all different, which is my point: there's a variety of schemes that all have well-understood pluses and minus, meaning new languages don't have to do the exploration themselves.

> The popular criticism is that Go isn't reinventing enough; that it's stuck in the 70's and it's not sufficiently innovative.

That's popular, but unfounded, in my opinion.

I find ironic that some HN commenters are criticizing the Go team for not being stuck in the past when they are precisely trying to innovate and questioning the status quo on package management.

> Only because the golang core team seem to insist on reinventing every wheel either without considering prior art, or at least without considering why prior art went with a particular approach.

Actually it's pretty clear that they did their homework.

If you count as “their homework” the dozens of iterations of this we’ve been through.

First, it was literally argued that just `go get` just grabbing `master` of all of your dependencies was good enough, because if you ever need to release something backwards-incompatible, you should create a new repo. I wish I were making this up.

Then we just vendored everything and never updated our dependencies, and that too was good enough.

Then GoPM, go dep, and a deluge of other tools that I don’t care to track down or sort into their appropriate order.

And now this, which decides to buck the trends that everyone else follows yet again and attempts to satisfy dependency trees by choosing the minimal compatible versions instead of the maximal ones, with the completely foreseeable consequences that:

  * security patches won’t be applied
  * issue trackers will be filled with bug reports that are closed with messages saying “this was fixed months ago, update”, and
  * updating dependencies will become *harder*, since this is a straightforward result of doing it less frequently

If you can't go get -u all on your project at least you have done something 'wrong' or your upstreams have done something wrong.

First, the upstream behavior; not supporting the published interface (as it was and should therefore work forever) means they need to deprecate that interface and release another with the incompatible changes. However the PROPER way of handling this is to add new versions of exposed public methods and interfaces / constants with new names and only if such a major change is required that you can't support the old interfaces, at THAT point, release at least under a new inclusion path if not new dependency name.

Things local a developer might have done wrong: A) include an 'immature' upstream repository that behaves like above. B) use non-public components or other unsafe practices to go beyond the interface contract that was exposed.

I guess using any dependency, directly or transitively, that declares itself pre-1.0 in the go or semver meaning of the word, is thus right out.

At that point, almost all of the go ecosystem can't be used.

Do you have proof that the Go team has not looked at any other dependency management approaches or considered ones from other languages?

Obviously I don’t, but the available evidence — to me, anyway — points to this being a cultural reality in the golang ecosystem.

I left the language years ago, but the steady stream of articles of the form “Why go doesn’t need $x” followed by innumerable comments, articles, and blog posts by people struggling with the lack of $x, followed by the go core team (due to the completely foreseeable consequences of not having $x) begrudgingly adopting $y, which is supposed to (but doesn’t actually) obviate the need for $x… I’ve reached my own conclusions, and I’m finding more and more people who are leaving the ecosystem with those same conclusions.

I believe that Go excellently solves problems that Google has. I don’t believe it solves many problems that most other users have, even though it might seem like it at the outset.

Wait until they finally cave in to generics being one of the top requests on Go surveys and bring something to Go following that same design process.

Reply on news.ycombinator.com