Show HN: ied – an alternative package manager for Node
github.com
github.com
I made this. I'm happy to answer any questions, but please bear in mind that this is a WIP. There is still a lot of work to be done, although feature parity with npm is not the goal.
Upcoming features are:
* Nix-like rollbacks * built in registry server * discovery + installation via BitTorrent DHT
Would love to get any feedback!
Also related: https://github.com/dominictarr/npmd
I've started `registry-mirror` to demonstrate how a Content-Addressed file structure and P2P discovery, can bring a lot of speed improvements, specially when the bottleneck is low bandwidth/latency to the backbone, by connecting to more local peers that have the content that we are looking for.
Right now, the goal with `registry-mirror` is to have a very large IPFS node in the network with the entire npm and that keeps replicating it, while end user machines only download the modules they need (and if they agree, provide them to the network as well). Each end user will be able to get the latest state of the registry, through a IPNS hash, a mutable pointer, that changes each time the mirror is updated.
The nix package manager model layers perfectly on top of IPFS' MerkleDAG (https://github.com/ipfs/specs/tree/master/merkledag), it can be a very awesome transport for ied. An example of a package manager that uses IPFS to distribute the packages is GX https://github.com/whyrusleeping/gx - Still a WIP.
If this is interesting to you, join us at IRC Freenode #IPFS, it would be great to bounce more ideas! :)
This is my biggest bugbear with npm, as it pretty much rules out doing shell scripting with node.
Having fast immutable shrinkwrap installs, and the ability to upgrade or selectively upgrade packages in the shrinkwrap vs. the semver in package.json would be a huge win.
In npm v3 updates the shrinkwrap as well as the package.json
I could see a case where I'd use `ied` to install deps for the speed, but continued to use npm for other features and interop with others.
This could be very useful - but would need shrinkwrap support along with the stuff on your TODO to make it viable for my current work projects :)
Therefore, npm and ied are compatible in the sense that you can use ied to install packages and npm for publishing them. At least that's the level of compatibility that is currently supported.
shrinkwrap is definitely coming :)
`ied publish` and `ied version` is coming next week. I'm also thinking of adding scoped modules, but I'm not sure about that yet.
You can also configure a private npm registry to be used: https://github.com/alexanderGugel/ied/blob/master/lib/config...
npm install 52.28s user 8.08s system 73% cpu 1:22.41 total
ied install 10.22s user 4.36s system 142% cpu 10.230 total
Very promising project!
Also the way it wires up dependencies (using symlinks) has been called "non humane design" by npm... so I thought it would be easier to just start a new project... it's not a lot of code/logic needed there actually... :)
I made a quick comparison here: https://gist.github.com/alexanderGugel/a10ed5655d366875a280
Basically ied uses symlinks in order to resolve circular dependencies, while ied exploits the fact that require "falls back" in the directory structure.
> Basically ied uses symlinks in order to resolve circular dependencies, while npm exploits the fact that require "falls back" in the directory structure.
In answer, why not create a new project? NPM INC controls npm, hasn't contributed it to the node foundation (despite playing a pivotal role in creating said foundation), and hasn't been especially good at taking contributions recently.
A new project dodges all those existing problems, demonstrates alternate approaches are both feasible and compatible, and destroys the myth that npm is fundamental to node, rather than simply the first of many package management systems that take advantage of node's import semantics.
Part of that is just the monstrous number of files involved. For example, one of our projects has 48 dependencies, which installs 24,421 files under node_modules. NPM could probably benefit from managing each dependency as an archive.
We do atomic deploys and try to make them reproducible, so for each deploy we do a fresh install from npm-shrinkwrap.json, but even when all the modules are in NPM's local cache it's very slow at copying everything.
Unfortunately, NPM doesn't version the node_modules folder (a package becomes ./node_modules/mypackage/...)), so you can't reuse it. The NPM cache is versioned ($cachedir/mypackage/3.23/...), but can't be used directly. It would be much better to skip the cache altogether, and have node_modules embed version strings (./node_modules/mypackage-3.23/...). Then you could easily share the folder across builds.
NPM is also pretty brittle. We frequently have deploys fail because of transient network errors (repository timing out or similar) that cause NPM to fall over. The dreaded mysterious "npm ERR cb() never called" error still hits us weekly.
(Speaking of reproducible builds: NPM lets people unpublish packages. Sometimes old versions just disappear, presumably because they were unpublished. 6 months later you want to deploy a certain app, and you find it depends on some package X, which deep in its dependency graph relies on package Y 0.3, but 0.3 is gone from npmjs.com, so you have to upgrade for no reason at all.)
Bootstrapping ftw.
This is a "cool" feature during development, since it's a nice proof of concept.
Originally I checked in the node_modules directory, but then reddit was shitting on me as usual (yes, you shouldn't check in node_modules in an actual app, but this is PACKAGE MANAGER!). As far as I know, npm has also its own dependencies check in + a ton of packages as tarballs for tests, so I might do that later.
To the OP: Don't let people bully you. Many people have strong ideas and will want you to do things their way. You aren't going to make everyone happy, though. Somebody will be pissed off no matter what you do (if you are popular enough -- normally people won't pay any attention to you ;-) ). "Because that's what I want to do" is a completely valid reason for any decision on your own project.
Finally. npm's node_modules makes Node on Windows unbearable from time to time.
Shut up and take my money!
In fact I literally just renamed it: https://github.com/alexanderGugel/ied/commit/84628b3c871c85d...
Originally it was called mpm, but I figured that would have been pretty confusing, but it looks like the new name isn't necessarily better.
I'm pretty terrible at naming. Any suggestions are more than welcome!
Only funny for anyone who hasn't been affected by one, or had friends or family who were. Still better to avoid names with negative connotations (and search for them first to check).
> Any suggestions are more than welcome!
A few ideas:
bpm - Better Package Manager
edge - the thing that connects nodes
jpm - Javascript Package Manager
ppm - Peer Package Manager
fpm - Functional Package Manager
ayp - All Your Packages
nnm - New Node Manager
Oh, lay off the holier-than-thou moral outrage. I bet you could mine for naming collisions in a space of three characters like this all day, or really all sorts of project names.
man 8 ss tells me it was written by Alexey Kuznetosv. I should strongly consider informing him of his trivialization of the NSDAP's atrocities.
That being said, it's a three letter name. It's very unlikely NOT to run into naming conflicts here.
edge - taken by Microsoft
jpm - JPMorgan
ppm - taken by Perl package manager: https://en.wikipedia.org/wiki/Perl_package_manager
fpm - taken by Effing package management: https://github.com/jordansissel/fpm
bpm - beats per minute
ayp - terrible to type, although taken by "Adequate Yearly Progress": https://en.wikipedia.org/wiki/Adequate_Yearly_Progress
nnm - What happens when it's no longer new?
Just in Germany for example there are a ton of companies called ISIS (just google "ISIS GmbH"). Being offended by a three letter shell command seems a bit over the top to me to be honest.
Edit: I won't respond to further comments on the naming issue. It wasn't my intention to name it after a weapon. As I said earlier, I will change the name as soon as anyone proposes a better one.
It isn't critical to avoid all possible naming conflicts, only to avoid 1) other command-line tools and 2) names in poor taste. Thus, only the conflicts with other package managers really matter.
I'd worry less about offending someone because they had to type it, and more about SEO. I'd stick with things that don't have pictures of gore and destruction on the first page of search results.
So, that's a really good reason to not.
Also, your response was shitty... You specifically state that you chose IED because it's "easy to type"... but when someone says "Hey, that's what we call bombs that insurgents use to kill people with", you reply "Yeah, but the alternative is an acronym used by JP Morgan... People getting offended by me naming something after a way to kill people are being over the top". If "IED" happened to actually mean something, fine, make a case... but it isn't actually easier to type than anything else... If you want that, name it ASD, which shouldn't have any conflicts, as the top google search result is Anchorage School District... and I couldn't find any conflicting package names.
I get it, people are constantly picking on things and suggesting that they need to be more PC... but in this case, there is absolutely NO reason for you to stick with the name IED... and several reasons to change it (SEO, offensiveness, typability).
Also, good news for german companies... The news has started using different acronyms for ISIS... I've seen IS and ISIL in regards to the paris bombings, as they're more true to the literal translation.
I think "bpm" is a clever name for a faster npm client because "beats per minute" is a speed measurement and thus associated with something going really fast.
IED on the other hand has the association of people dying or being crippled by terrorists.
Say the programmer has english as a second language, its really easy not to know "common" things like this. Taking outrage in these cases is not in very good faith as the presumption is that the name is the same thing as the acronym in question.
Lets say they chose to name it baa, and people reacted to it thusly:
- what is this implying all users are sheep! needs to be changed/boycotted
- ...
- this makes no sense, name it after its function xyz.
- this is a horrible name, I'm going to encourage everyone I know not to use this
Or "leaf" for singly-connected nodes. Or "graf" for that matter.
Super easy to type, and available on npm. https://www.npmjs.com/search?q=pkn
Using many dependencies also makes your application a fractal of dependencies. Library you use has its own dependencies, which have their own dependencies, which have... And so on. If you happen to want to put your application to DEB/RPM packages properly, host your app's dependencies locally to isolate from network outages, or do virtually anything non-trivial with your app, you hit the list of dependencies very hard.
It's not to say dependencies are evil; like any generalization, this would be simply stupid. But they have their cost, and it's large. You'd better make sure a library really simplifies your life before adding it to your project.
I find that using tools like npm, nuget, gems, and the like along with github for core libraries makes managing dependencies FAR better than relying on an internal copy of said package in my project, which is just as risky as an internal fork. Yes, there's a chance the package system can be down while you are wanting to deploy.. that's why you prepare on a staging server, and duplicate that to production. Hell with things like docker, you create your container, and run it with environment variables for which environment it's running against.
Of course it does, but again: it's you who controls the code, not the library's upstream. You can easily patch and adjust whatever necessary, without being bothered with future merges from upstream.
This aspect grows in importance as the library gives less and less benefits. For example, I wouldn't hesitate to use Flask or Django for a web application, but I would think very hard what good is going to give me logging library other than Python's built-in.
> I find that using tools like npm, nuget, gems, and the like along with github for core libraries makes managing dependencies FAR better than relying on an internal copy of said package in my project
npm and gems give you dynamics very different from seeing dependencies by yourself. You see dependencies as very cheap (which is not true, it's just the cost is hidden from you), so you don't bother with thinking if you really need the new dependency. If it can shorten your code by twenty lines, you see it worth.
I am not particularly sensitive to these things, but upon reading the name, an image came to mind of a soldier leaning against a concrete wall on a dusty urban street, leg recently blown off, bleeding out, in shock. That image is now permanently associated with your project for me :)
For comparison, imagine if you'd called it "iud". At least in American society, this is more readily agreed to be offensive because of our sex taboo and gender inequality.
For a lot of people, the thought of war evokes deep visceral feelings of horror and fear. Such people will be less likely to use your project, which you could argue is a form of discrimination, and in any case doesn't benefit anyone.
spend no more than 2 minutes picking a name, and stick with it
'$ package install ...'