Comment by tmcw | Hacker News Reader

Comment by tmcw | Hacker News Reader

How to write a silly post about JavaScript dependencies:

- Don't mention other languages, lest you reveal that most of them have similar dependency bloat problems.

- Talk about devDependencies and dependencies without considering how they might be entirely different, say… one kind gets bundled, the other doesn't.

- Always use npm, so you can highlight duplicates. Don't use yarn.

- Adopt a narrow definition of 'dependency hell' so you can focus on the thing that JavaScript does imperfectly (minimizing number of dependencies) and avoid talking about the things that it does well (handling conflicting deps, solving diamond deps, resolving dependency chains)

- Don't worry about explaining how any of the issues you're discussing have real-world implications. Have companies been failing because they run out of disk space because of node_modules?

- Take automated security audits extremely seriously. A backtracking bug in the argument parser for the web framework's local CLI tool? Of course it's not exploitable, when… it's a web app, and the CLI tool is something you run locally. But add it to the count as if it is!

> - Don't mention other languages, lest you reveal that most of them have similar dependency bloat problems.

The post contains a graph with the package counts for other languages [1]

1 https://d33wubrfki0l68.cloudfront.net/7481900a584f733e7e9db7...

Is node's biggest crime is that it's easy to publish packages? Is this an argument for intentionally complicating package systems?

The issue with NPM is that packages tend to be more granular than other languages.

For example: The npm package "is-even" depends on the "is-odd" package. (In case it's not clear, this is a real package, not a made-up example.) In most other languages, these would be bundled together into an Integer Utilties package, if they're not already part of the standard library. In NPM, they are separate packages.

I believe there is, or was, a technical reason for NPM to have very fine-grained imports. Perhaps something about not supporting partial imports? I can't exactly find it. But the point is: NPM having more packages does not translate to a larger and more vibrant ecosystem. In some cases it simply means accomplishing the same thing requires a larger dependency graph.

That's not to say it's inherently a bad thing. But that's at least why it's not inherently a good thing either.

[1] https://www.npmjs.com/package/is-even

> In most other languages, these would be bundled together into an Integer Utilties package, if they're not already part of the standard library.

This is the root of all the many issues. There is no standard library. So things that are trivial to write but also dumb to rewrite every time you need them (left-pad a great example) turn into tiny dependencies.

> For example: The npm package "is-even" depends on the "is-odd" package. (In case it's not clear, this is a real package, not a made-up example.) In most other languages, these would be bundled together into an Integer Utilties package, if they're not already part of the standard library. In NPM, they are separate packages.

People should not be using these functions at all, in any language, in a standard library or otherwise. To convert the even/oddness of an integer to a boolean in Javascript you can use !!(n % 2), which returns true for odd inputs, and false for even inputs. If you want to be extra explicit you can temporarily store the result of that expression in a meaningfully named variable. If it will just be used in a conditional, then the !! can be skipped.

Depending on a third-party package for this kind of thing is insanity. But the blame lies with programmers who use this package, not with the programming language.

If the standard library includes a variant for these or any other functions, it's probably idiomatic to use them. They handle edge cases, are better optimized and they're explicit.

There is no way that any Javascript library function is "more optimized" than just using `n % 2`, which is also plenty descriptive.

If this is very performance sensitive you can try using the somewhat more obscure (because many people have minimal experience with bitwise operators) `n & 1` instead.

Given an integer input, thare are no "edge cases" to worry about here. If you expect floating point inputs, you need to think carefully about what the correct output should be anyway. A library isn’t going to save you.

(If your "edge case" is that someone might want to know whether a string is even or odd... well, you have other problems.)

It's funny, because in the case we're talking about, there appears to be edge case behavior if the number is a "safeInteger." While I agree in principle I would never import something as granular as "is-odd," just researching this specific package revealed an edge case someone ran into and solved, and you don't have to worry about as a dev if you take it off the shelf. https://github.com/jonschlinkert/is-odd/blob/master/index.js...

Your "isodd" function should probably not be doing bounds checking and throwing an error when it encounters large even integers.

If you have a use case where you need to know if you multiplied two large integers and got a result larger than 2^53, then you should do it explicitly before you get to the point of checking if your number is even or odd.

These comparisons are so bad.

NPM is hands-down the most popular packaging ecosystem online. So of course it has more packages than other ecosystems, more developers use it. It's not a good comparison.

I would not be surprised to find that JS bloat tends to be more severe than other languages, because Javascript makes it a lot easier to import and publish packages. I would also not be surprised if Node package management ended up being a bit worse, again because Node lowers the barrier of entry for inexperienced programmers to publish packages on impulse, even if they don't plan to maintain them. And its just undeniable that Javascript is a heavy intro language, so the average developer quality (especially around responsible dependency management) is probably less than other languages.

But the package count stats need to just die. When we talk about dependency bloat, useful stats would be:

- What is the average/median number of (deduplicated) dependencies in a Node program vs a Rust/Python/Ruby project?

- What is the average/median number of lines of code in a Node package, minus its dependencies? How does that compare to the average/median package in Rust/Python/Ruby?

- What is the average/median number of outdated dependencies in a Node package, vs the average/median in Rust/Python/Ruby?

- What are the average/median number of different authors across a Node package's dependencies vs Rust/Python/Ruby?

- What percentage of the Node ecosystem actually gets used? Are we looking at a long tail of mostly ignored packages, or is usage fairly spread out and diverse?

Heck, even this mostly useless graph would be better if it just adjusted for the number of users for each platform. It's pretty tough get that data, but there are sources you could look at to try and guess, including StackOverflow's dev surveys[0].

The "how many packages are there" metric means nothing when it's quoted in isolation from other data. It's like claiming Windows developers are vastly more productive than Mac developers because more Windows software exists.

[0]: https://insights.stackoverflow.com/survey/2019#overview

Also keep in mind it's far easier to publish on npm and that's a good thing.

It literally takes days to setup shit to get published on maven central and even then you can't set and forget. It's a pain in the ass.

Yesterday I had to use the web fullscreen API. Didn't work on an iPad, so I installed npm package screenfull that handles browser incompatibilities. Works like a charm.

These kind of things are specific for JavaScript because of the mentioned incompatibilities, and of course increase nr packages used. But I'm very happy they exist.

They also strategically avoid mentioning the payoff which is that your project is entirely self-contained. `node_modules` is at once your complete development kit, build tool, compiler, and collection of runtime dependencies.

Once you check out a node project with a package.json you can reasonably assume you can install it and run it anywhere. A docker image for a node project should just be "install node && npm build". You're side-stepping the entire dev ops nightmare. Find me another language with JavaScript's popularity and that level of simplicity. I'd go as far to argue that this is what happens to any massively popular language that has a functional package manager from day one.

I'm totally baffled by this statement.

Find me a mainstream language that's more complicated than this? "Checkout and build" works pretty much everywhere? I mean, you need the right version of mvn or pip or whatnot... just like you need the right version of npm.

If anything, node/npm deserves special mention for NOT having reliable repeatable builds. Part of this is that a lot of packages rely on native code. But most of it is that the ecosystem culture defaults to "always grab the latest versions of everything".

And npm only got package-lock.json a few years ago, not "from day one". Prior to package-lock.json, builds were wildly unpredictable - like, expect breaking changes week to week even if you don't change anything at all in your code.

If you want an "entirely self contained" payoff, languages that produce static binaries are pretty hard to beat. Node is not that.

Heck, in javaland the paractice now is to put the build tool in the repo, as you will only update maven once in every decade git won't complain too much about the binary, or you can just update the version in the wrapper and put the binary on the git ignore. First time you run it, it will update the build tool.

Nowadays on javaland you should only need to have JDK and git/svn/hg/(wtv version control you use). And with the JDK, you can always download the most recent that java is backwards compatible.

Yes and now lock files exist?

Are you actually arguing over the state of JS development from 4 years ago?

I have pointed out that, contrary to the parent, Node is not special, and certainly wasn't "from day one" as claimed.

To put it simply to build a modern Java project you need the jdk + bazel + docker + maven + runtime classpath for a sub-set of npm's features (as far as I know you still can't include two versions of the same maven dependency).

For a Node project you just need node. npm is part of node, and yarn is optional. The repeatable builds was a real issue but that's been solved years ago.

That's not true for Java. Most require Maven and then it's mvn install (you need the JDK for maven, but that feels similar to saying you need node for npm, they effectively come together).

Npm's dev dependencies are analogous to Maven's plugins.

Bazel is an alternative to maven. Docker fulfills the same role as in the Node ecosystem. The runtime class path is an implementation detail that you need to concern yourself with if you wish to run your Java code without Maven (analogous to running your JavaScript code without npm).

The two forms of the same dependency is a limitation that stems from Java the language rather than its ecosystem.

FWIW, nearly every language ecosystem I know of is at the point of "download build tool, run build tool."

Worse than that: lock files aren’t lock files the way that they are in every other language package manager that has lock files. In Cargo, Bundler, and Mix, you specify a pessimistic version (~> 2.1) and you may get 2.1.1 or 2.3.0. But that version is _always_ the same for every developer because the lock file locks the version and you explicitly upgrade after that.

I recently had a case where a developer joined us on a project and he got a different version of a package than I did because the lockfile didn’t constraint the dependencies and sub-dependencies and everything else. (For that you have to pass an explicit parameter like `--ci` or `--frozen-lockfile` depending on which of three different package managers you use.)

Bollocks, I say.

> A docker image for a node project should just be "install node && npm build"

That defeats the purpose of having separate dev and runtime dependencies and results in a huge docker image filled with dev-time dependencies.

Also, other package managers do this fairly well. Java has maven and uber-jars (for self contained artifacts), python has pip and venv, etc.

Unless any library you use depends on a native module, then you are in for surprises.

How to downplay the severity of the problem:

Ignore the ridiculously small functionality scope that acceptably qualifies for a js package in a way that does not exist in any other ecosystem and ads a big multiplier in front of all the other risks involved.

If these claims were true, we would have seen some examples by now. Instead, we get some tired invocations of "leftpad!" along with some more tired FUD. Who has been hacked because they used javascript instead of some other "more mature" language?

There is more to dependency risk than 'getting hacked' - correctness, maintainability, IP due diligence all play a role.

Yes. I've had several apps that can basically never be upgraded due to complete insanity deep in their dependency trees.

I'm pretty cautious about third party dependancies, but some of those libraries' libraries are not, and it's really hard to see that coming.

If you expect to live with code for 5+ years (and I understand that's not everyone, but it's almost always me), this is a big problem.

We've all lived with code that was 5+ years old, even if not all those years were ours. b^) Often that meant wrangling envars to point to the right ancient binaries, assuming we could somehow get those binaries installed. None of that is required with javascript. If a dependency is causing problems, just replace it with a different one. (npmjs.com has lots of packages!) Or, write something yourself. In five years you'll find enough spare time.

> - Don't mention other languages, lest you reveal that most of them have similar dependency bloat problems.

no such problems in C, i promise you

Reply on news.ycombinator.com