Comment by aaronkaplan | Hacker News Reader

Comment by aaronkaplan | Hacker News Reader

Parent Full thread

I think you're right that in principle one can make a maven build deterministic, but in practice I have yet to see it done. Of course, that could just be a statement about the crowd I hang out with, but I think there are more fundamental problems. Wouldn't you have to specify explicit version numbers for not only your immediate dependencies, but also all transitive dependencies? Not that that's necessarily a bad thing in itself--regardless of what build system you use, if you want a reproducible build, one way or another you have to specify particular versions of all dependencies. But if you have to provide a configuration file that explicitly lists all direct and indirect dependencies, then what's the point of all the sophisticated transitive dependency management functionality we hear so much about?

Also, you didn't address the gp's complaint that the build tool will spontaneously update not only the project dependencies, but also itself. Some maven plugin developer out there can release a new version, and a project that compiled fine yesterday suddenly won't compile anymore. I'm sure there's a way to force it to use particular versions of the plugins as well, but by the time you explicitly list every transitive dependency and every plugin used in every phase of the lifecycle, you've got a really big configuration file, especially for a tool whose motto is "convention over configuration." Not to mention that if you accidentally forget to specify a version number for one of these potentially dozens of packages, maven will happily default to using the latest version, which makes your build secretly nondeterministic.

What you're missing here is sort of described in the RTFM. There are a couple very good, free books on maven that describe this, but in essence:

1) You absolutely should have your own local repository (e.g. Archiva). 2) You should use <dependencyManagement> and <pluginManagement> tags to force all projects to have the same versions o all deps and plugins. 3) Don't even try using snapshots unless you know what you are doing.

A local repository doesn't address the problems I mentioned above, it addresses a separate problem, namely the nondeterminism that comes from relying on external repositories that might not always be available. As for the <dependencyManagement> and <pluginManagement> tags, you didn't address my complaints: (1) if you have to explicitly list every dependency, including indirect ones, doesn't that negate the benefit of maven's transitive dependency management? [Edit: ok, not entirely, because it takes care of downloading the dependencies automatically. But one of the supposed arguments for maven is this: if your project depends on foo, and foo depends on bar and baz, you can just specify foo in the pom, and bar and baz will be pulled in automatically. But that's disingenuous: if you want your build to be deterministic, and of course you do, you still have to specify bar and baz explicitly.] And (2) if you forget to specify a version number for one of the dependencies, doesn't maven just default to the latest version at build time, thus becoming nondeterministic?

... if you forget to specify a version number for one of the dependencies, doesn't maven just default to the latest version at build time, thus becoming nondeterministic?

No.

... if you want your build to be deterministic, and of course you do, you still have to specify bar and baz explicitly

If you declare a dependency on non-snapshot foo, then it will also declare a dependency on non-snapshot bar, ergo, reproducible build.

Your comments seem to be the standard "I don't understand Maven, but I hear that it's broken."

The original complaint was

> the far more likely scenario is your project depends on a specific version of some other project which in turn depends on the LATEST version of some other project, so you still get hosed even when downstream providers do remember to bump versions!

so are you saying this is impossible (non-snapshot artifacts are somehow forbidden from depending on snapshot artifacts, which is not what was claimed), or that every author has to understand this and package their artifacts perfectly to prevent this from becoming an actual problem?

Yes it does. A good local repo will cache it. They act as mirrors. Once its cached, and your build is working fine, you can force all your developers to use only your local repo. This solves all non-determinism problems by allowing you to have complete control over all version of all deps and plugins.

Take a look at how to configure settings.xml to force your users to use only your local repo.

As far as <dependencyManagement> tags -- the point of that is to hard-code the version of foo in one spot, like this:

<dependencyManagement> <dependency> <groupId>com.example</groupId> <artifactId>foo</artifacatId> <version>1.1</version> <dependency> </dependencyManagement>

In child pom, you then have this if you need foo:

<dependency> <groupId>com.example</groupId> <artifactId>foo</artifactId> </depdendency>

And if you use something like Archiva, and force your users through it via settings.xml, then the first time someone downloads foo, Archiva gets it from the web, caches it, all other requests for foo go to Archiva. If foo has a transitive dependency, it will also get fetched from the web one time and cached in Archiva. Thus, guaranteeing that your build is deterministic -- unless you explicitly change foo/bar in Archiva yourself.

Reply on news.ycombinator.com