Hacker News Reader

Comment by devwastaken | Hacker News Reader

This is something I've always been concerned about with the node/NPM environment. Any project, even the smallest ones, have hundreds of dependencies. All it takes is some small lines of rogue code, and your entire project is vulnerable. Especially in JS, where you can do network requests and various critical actions all in one line of code.

View on HN

The `left-pad` debacle a year ago[0] should've served as a wake up call to people writing projects with hundreds of dependencies.

[0]: http://left-pad.io

It's next to impossible for the average team to avoid hundreds of dependencies as their project grows though. You could write your own everything, but then what's the benefit of the node ecosystem?

Edit: spelling

How many of those dependancies are trivial though? It's one thing to rewrite express, but something different to rewrite a package that just contains a function that capitalizes the first letter of a string or something similarly contrived, yet plausibly already a package.

That's the argument behind a stdlib of sorts for npm with blessed packages that are actively maintained and signed. Having every project rely on tons of dependencies isn't great, but re-writing everything a bunch of times also isn't great. There are bound to be bugs within 100 different implementations of X.

It is certainly out of control though. I just checked the node_modules of a fresh create-react-app generated app and there are 877 packages! Tons of duplication here, like having both array-uniq and array-unique (not to mention that's a feature built into languages like Python).

I wish the node ecosystem would learn from more mature ones.

I would like to see a curated set of popular libraries that are stabilized and blessed, and a core group that handles security updates and upgrading packages in the blessed set.

There are attempts like https://nodesource.com/products/certified-modules but do you trust the ratings if they don't have control over those modules?

who gets paid to support that ecosystem, and who does the paying? as a simple example, I use the hapi.js server framework (because it is awesome). At one point, it was supported by Walmart Labs. It isn't anymore - it's mostly supported by the original author (Eran Hammer), who is getting paid less than $5k per month to develop / support it via patreon (and take a look at the bimodal distribution on his patreon page). And that's just for part of the overall ecosystem.

Part of the excitement of js dev is that there's always really useful libraries being created and distributed (ramda, rxjs, react to just name three things that start with R).

Not sure there is a good solution here - we want tons of value, but I suspect nobody's willing to actually pay for it. Libraries like cross-env, ramda, and so on all are excellent, useful, well-written, and the authors are responsive.

Other programming ecosystems have solved these problems without these glaring problems, so I don't understand why you're acting as though this is unsolvable while requiring lots of money.

Other programming ecosystems are at least an order of magnitude smaller and not intrinsically tied to technology that's in relatively extreme flux with a massive amount of users (browsers).

least an order of magnitude smaller

No they aren't.

It's true that NPM is the biggest package repository by some way, but it's only ~2x the count of the Maven repository (492135 vs 194954). PHP and Ruby also have the same order of magnitude.

tied to technology that's in relatively extreme flux with a massive amount of users

Maven/Java has similar challenges to some extent with both Android and server side development being extremely common in the same language. The large number of users and the extreme technology flux is also similar.

Name one.

Java/Maven.

So you use https://github.com/whispersystems/gradle-witness, and you manually verify everything you whitelist right? And you never typo, right?

I doubt he does, but i also doubt that installing a framework via maven brings in 1,000 implicit dependencies from 1,000 untrusted authors, unlike in JS land.

Edit: react, webpack, babel, babel-preset-env bring in 1,257 dependencies. Try vetting all those by hand.

I think my (primarily Java) project at work brings in about 50-100 external dependencies. That includes some ridiculously large frameworks, like Spring. Honestly, I don't know because I don't have dependency problems.

You know how hard it would be for me to vet the dependency if I did have a dependency problem? "mvn dependency:tree > deps.txt." It would take half the day, but I could vet them all.

I also have a Django project on the side. It has 8 dependencies. I have vetted all of them very carefully. It's smaller than the work project, but that's not the reason why it has fewer dependencies than a typical JS mess. It has fewer dependencies, because Pythonistas have a philosophy and cultural practices that produces quality software with lots of functionality included.

Npm being split means more metadata, not necessarily more code. You admit to not reading your code or inspecting your dep, and you assert that you have no problems - based on what?

Your deps.txt is barely different than a lock file here.

Reading the code from dependencies is not really hard anywhere here.

> you assert that you have no problems - based on what?

The software works as intended and I know that my dependencies have not changed without me explictly changing them.

> Your deps.txt is barely different than a lock file here.

That command produces a tree of all dependencies and their dependencies. I don't know what that has to do with a lock file.

0. What I do is irrelevant to whether or not the other people in the community have solved that problem. Which they have.

1. That's a Gradle plugin. I do not use Gradle. I use Maven.

2. I don't need to "manually verify everything I whitelist" (which is not exactly how Maven works but w/e) because everything in Central is cryptographically signed and can't be replaced simply because someone deleted all of their projects in a fit of pique, and some other person swooped in and used the exact same name.

3. I don't have to worry about typos, because I can't add a dependency to my project without editing the project's POM, so fat fingering on the command line cannot add random packages to the project. Plus, Maven artifacts are namespaced, so I would have to be pretty drunk to make so many typos that a different, existing package downloaded.

2. If you don't whitelist your deps, then your dep chain can change with any update. Signing keys prove only that the publisher held the keys, not that they aren't malicious, or that the keys aren't stolen.

3. Fat fingering your pom is no different, this argument is as wasteful as 0 and 1 and you know it.

And while we're at it

https://issues.apache.org/jira/plugins/servlet/mobile#issue/...

Also, if you read the docs you'll notice that the deploy plugins encourage users to put their central passwords in an xml file in the plain AND their pgp key passphrase in another, in the plain. No joke. Read the docs.

An injection attack on a download is not one of the problems we're actually talking about here. That's a problem that affects any package manager.

And it's patently false to say that the deploy plugin documentation encourages you to include that information. They very clearly preface that with "if your repository is secured" and use the explicit example of an internal repository within a corporate context. People smart enough to read that documentation in detail should also be smart enough to determine when it is and isn't a good idea to do that.

I didn't say that "I don't whitelist my deps." I said I don't need to manually verify every tiny thing that I whitelist.

If you actually knew what you what you were talking about, you would know that you can't specify a dependency in a Maven POM without specifying a version, so everything is effectively version pinned and doesn't magically change unless you do it explictly. And you can't change an artifact after it has been released; you have to issue a new release with different coordinates. These are features of Maven that npm is missing that would make it vastly more secure than it is now.

And fat fingering a pom is different, because it requires many more mistakes to download some other software package than the one intended. That's what this discussion is about right? That it's absurdly easy to download the wrong package on npm but not other package managers, because npm is an insecure package manager.

Maybe the benefit of the node ecosystem isn't as great as you imagine if a project requires hundreds of dependencies when projects that are part of other ecosystems only require dozens.

Reply on news.ycombinator.com