Hacker News Reader

Comment by agumonkey | Hacker News Reader

care to explain ?

I have hate similar levels of hatred as OP toward java packages for years, then with age I started to appreciate the abstraction (independence from the file storage name space etc) but still i find clean python so much nicer 80% of the time.

I'm really curious about your point of view.

View on HN

As someone moving from java to python recently, I'm more flabbergasted by python's dependency management than I've ever been with java's. "Classpath hell" hasn't been an issue for almost the last decade in my experience, and things generally just works. Maybe because the big dependencies everyone uses moves slow and are great at backwards compatibility, but upgrading something rarely never breaks, so seldom an issue with two different versions of something.

And modules have solved the issue of "you're actually using something internal from our package that might break in the future".

But in python upgrading something minor quickly wreaks havoc. Everything depends on different versions of something, which are quickly incompatible. So have to pin and do lots of manual testing and fight with pip for a while. Very easy to import and use stuff not supposed to be part of the packages' public api etc. And I find it even harder to know what's actually in use between the global packages, the virtual environment etc., than just looking inside my jar.

I feel your pain. Check out poetry if you haven’t seen it already as a modern alternative to pip.

Given a class and its package, I know exactly how to find the original code or class file.

Using the classpath, I can specify exactly the search order of classes, including overrides and whatnot, when the VM searches for them.

Need to debug a Jar? Unzip it (it's just a Zip file).

Jars can contain non-code resources, essentially allowing you to run an entire app using a single file, without the need to first decompress to the filesystem. Java gives you great utilities to find those resources.

Classloaders can be customized and you have complete control over how the JVM sees code.

It's boring, to the point, and simple to understand. I don't get how people have so many issues with them...

But if it is a class that exists in multiple jars in different versions due to dependencies dependencies depend... then it is not so easy when the wrong class is loaded.

That is why application servers use multiple classloaders isolated from each order, and in stuff like WebSphere you can even specify which overrides which.

I am not familiar with websphere and it is long time since I have used weblogic. Maybe it has the same feature? But it sounds like a tedious process to go through when trying to resolve such a problem. Though it also shows how serious the problem is. That the Application Server vendor builds such a feature.

All languages suffer from some form of "dll hell" when everyone goes through endless third party dependencies, the major difference is how those conflicts get sorted out.

The solution in the Java world allows kind of shipping both conflicting libraries and still have a go at it.

Other ecosystems might make you rethink the whole dependency tree instead.

I guess it was similar, I never used WebLogic.

https://www.ibm.com/support/pages/system/files/support/swg/s...

Classpath isolation is part of the JEE (and now Jakarta EE) specification for application servers. This guarantees that the application servers can host multiple applications without the classes in one application bring used in another.

More concretely, if one application packages log4j 1.2 and another packages log4j 2, the application servers guarantees that the two log4j libraries are isolated and do not interfere with each other.

Tomcat provides isolation between multiple webapps on the same Tomcat instance using a classloader hierarchy [0]

Barring the system classloader - where one ought not to place anything unless in dire circumstances - one gets very nice class separation.

Infact, every JSP is compiled to a Java servlet and then loaded in its own classloader. To see the speed - edit a JSP page and then see how fast the change is reflected. When devs edit JSP pages and see changes, a fresh classloader picks up the new class file and the earlier classloader is discarded. Due to classloader inheritance, the new Servlet is able to access Java classes from the parent classloader (the webapp).

For production deployments, and ideally for continuous integration setups where one should compile once and test and use the artifacts everywhere, one can use the pre-compilation feature [1] where the JSP pages are compiled as servlets during the compilation process itself (and not at runtime on production environments).

[0] https://tomcat.apache.org/tomcat-8.0-doc/class-loader-howto....

[1] https://tomcat.apache.org/tomcat-8.0-doc/jasper-howto.html#W...

A huge disadvantage is that you can not have different versions of the same dependency used across the project - if you do it will, AT RUNTIME, fuck up your application (since Maven's dependency resolution is very primitive).

But then you have the ability to exclude dependency X's dependency 1.4.0 because you're already using 1.4.8 of Y, and 99% of the time, it works.

And I've had so many times when I've wished I could do this for Python dependency trees.

Sure you can, you just need different classloaders.

I guess the span of my coding is really really small, I never really needed any of these.

to me the issue is that it creates parallel graphs on top of the FS, and that people, unless spending some time reading the spec (oracle basic doc website was utterly useless for me), will have barely no visibility on.

what I appreciate is that it's fully standardized in the spec, so you don't have to deal with competitive packaging systems / hygiene like in other languages.

> and that people, unless spending some time reading the spec (oracle basic doc website was utterly useless for me), will have barely no visibility on.

It's literally modeled after PATH, which behaves exactly the same and has been used just fine for decades. I don't really see the argument here.

A classpath is more complex than a fs path right ? there's more nesting levels. $maven/.../foo.jar contains stuff I don't know, and the graph is also not something necessarily obvious. Or maybe I'm just too limited and confused (plausible)

It's not really more complex. It's just a search path. There's no graph. It goes entry by entry, "can I load the requested class from this location? nope, okay next." and so on, finally ending up with a ClassNotFoundException or whatever if it can't find it in any of the classpath entries.

Not really any different than PATH.

PATH lists folders with files (or links) it's pretty much flat

classpath lists folders with jars which embeds their own subtree of classes, that's one difference in my (poor) understanding

then, as far as i can recall, each class file can define package namespace so looking at the class path I have no clue what is there or what isn't so classpath errors felt more daunting to me

now that the JVM does a tree search over all of this in an obvious linear manner, i can understand, but as a user i still find classpath and armies of jars very hard to map in my head (added to that the verbosity and large number of classes in many java apis)

if i'm still off, then there's nothing saving me :) except maybe if you have an article about classloading and all that, that may remap my brain

> classpath lists folders with jars

No, the classpath lists folders of classes, or actual paths to jars. Not paths to folders of jars.

> then, as far as i can recall, each class file can define package namespace so looking at the class path I have no clue what is there or what isn't so classpath errors felt more daunting to me

Classes are resolved from the root of the jar. Just inspect the jar to see what's in it. It's not common for third-party libraries to have conflicting package names.

I don't know why you're talking about tree searches.

The JVM doesn't look for classes in jars unless you explicitly specify the jar on the classpath. It really is just like PATH if you imagine that a jar is simply a compressed folder (which it is).

Using Gradle for local stuff takes pretty much all the pain out of it, and when running in production I just have a single .jar file with all the dependencies baked in. I used to get annoyed by classpath issues but since moving to Gradle I haven't had any.

Yep, this is also what I've experienced. Gradle isn't a "good" build system, by any measure. But it gets the job done and can handle the Java story quite well.

I mean, python is a lot cleaner than java in a lot of ways, but the PYTHONPATH environment variable does a very similar thing in a very similar way.

Reply on news.ycombinator.com