Hacker News Reader

Comment by coding123 | Hacker News Reader

What really sucks is people committing node_modules, that's just plain wrong.

No what sucks is they made a clickbaity article by not removing checked in dependencies right at the beginning.

There are even scenarios where I’ve seen Java projects check in their dependencies (in conservative industries) so this erodes the value of the numbers greatly.

We have a pretty big issue with duplication of effort. I was hoping for an article that would be a wake up call but instead it’s just measurement artifacts.

Copy-and-pasted libraries are a direct cause of duplicated effort.

If you all grab it from the same repo (possibly through some sort of 'repo link' mechanism that keeps a local clone but is clearly marked as a clone, of course) then when a bug is fixed, it's fixed.

If you copy and paste it, suddenly every fix has to be done once per project using the library.

No they aren’t. They’re just a snapshot that works when the internet is broken, or when you can’t trust the internet.

Nobody is supposed to modify those files. They’re just a local replica.

This reminds me to hate AWS Lambda. I am not sure if anything has changed since my last use (around spring of 2017). I wrote my code in Python, and wanted to use psycopg2 to connect to my PG RDS instance. Luck out, yike! I was told to commit some psycopg2 files in my repository. I don't remember whether s3 was a feasible option, I think it wasn't; even then I would hate having a separate place for storing dependencies just because AWS couldn't solve customer problems.

And no, I refuse to believe such requirement is hard to solve at all. I refuse to believe AWS doesn't create some container-like environment behind the scene for launching and running a function.

See https://forums.aws.amazon.com/thread.jspa?messageID=791221&t...

> that's just plain wrong

there was at least one point in time where that was the recommended strategy

I'm not sure I could understanding the reasoning behind that. Does it have to do with dependency versions or the assumption npm might not be available or what?

There were/are a few factors. NPM availability is definitely one - before caching, and without the overhead of running your own npm replica. It also didn't used to have things like lock files. Vendoring gives you a deterministic build and removes availability concerns. In that aspect, it's not the worst thing ever, mostly just leads to noisy diffs (and maybe c extension issues if your team works on a variety of OSs?)

This is pretty much what the golang world does (though now there are some tools that do a better job).

Both actually. Even with yarn and lock files, npm servers can (and eventually will) pull a rug from under you.

I have already been in a situation where a dependency version that I was locked to was simply removed from the official registry.

The right solution is to have your own registry or backup the archives of dependencies that you are using.

I think it is better to commit the archives rather than the whole node_modules as it does not produce a mess.

Yeah running a registry is the better option, although another solution is forking / committing the dependencies in a separate repository, and use branch + shallow clone to keep the download size small.

What I ended up doing at work is to have a docker image of a registry with package zips committed in the repo (with a small script to publish all packages in). That way I can rebuild a running registry in a few moments.

like this one?

https://evertpot.com/npm-revoke-breaks-the-build/

... until LeftPad 2.0 [1] happens, then everyone will be mocking those who didn't commit node_modules.

1. http://blog.npmjs.org/post/141577284765/kik-left-pad-and-npm

Seriously. I've been on a deployment that failed because China blocked access to Github for a little while over some piece of code in some repository. Make sure your dependencies are available.

What's wrong is that the package manager and repo manager aren't talking to one another. "Keeping a copy of the dependency with your code" and "copy and pasting the code" should not be the same thing.

I agree, ideally artifacts should be stored and cached, e.g. in a stored container or as a tarball.

Other than a local npm mirror, what do you think is the best way to store artifacts for new builds? Vendoring code is not a bad solution if it permits new builds to occur even in the event of a package manager outage. I see vendoring code as the simplest solution for small companies who don't want the management overhead of running a local mirror of package managers.

That said, I'm very much open to learning a new technique for handling this problem.

Reply on news.ycombinator.com