The problem with Pypi is the transitive dependencies. You may trust a high profile package but it could pull in any number of projects that are exploited by bad actors.
Unfortunately, it's not an easy problem to automate the detection. There are some heuristics, but you're basically writing something like Norton Antivirus for Python packages lol.
A sufficiently skilled attacker can make their code look totally innocent[0]. :)
Source: Former security engineer at Uber/Snap that's now working on tooling in this space[1].
0: "Underhanded C Competition" http://www.underhanded-c.org/
1: Open Source "Supply Chain Security" platform https://github.com/lunasec-io/lunasec
The only company that has a real chance at making a dent in the issue is GitHub and to a degree they already have.
SOURCES:
- https://www.mandiant.com/resources/blog/supply-chain-node-js
- https://www.zdnet.com/article/backdoor-found-in-ruby-library...
- https://www.reversinglabs.com/blog/mining-for-malicious-ruby...
- https://news.sophos.com/en-us/2021/10/24/node-poisoning-hija...
- https://www.bleepingcomputer.com/news/security/dev-corrupts-...
- https://portswigger.net/daily-swig/amp/malicious-python-libr...
A high level python package is more likely to depend upon 3-30 well known packages.
Npm packages have really crazy dependency graphs. It's usually on the 1000s of times larger than what you get on most languages, 100s of times larger than python or ruby.
Hence left-pad. And this pet-peeve of mine https://nodejs.org/docs/latest/api/process.html#processargv (who cares about the node fullpath in argv[0]?)
The problem with left pad was a security one, not a package bloat one.
> It is highly probable that the majority of current string padding implementations are inefficient. Bringing this into the platform will improve performance of the web, and developer productivity as they no longer have to implement these common functions.
“It’s too trivial” is not why left-pad was in the news. Go read about it to understand the actual issue: https://www.theregister.com/AMP/2017/07/12/javascript_spec_s...
Of course it was useful, hence why most non-crappy languages had it in its stdlib from inception pretty much
But building a package just to do left-pad is stupid, especially since it can be implemented in a couple of lines
Nobody thinks leftPad was not a useful function. The question is, was it useful enough to counter all the risks of npm, probably not. In the stdlib there is no such risk.
My point has been this whole time that left-pad was not a story of a trivial function needlessly pulled from an external source as the person I replied to had claimed, and it appears you agree. Good!
But, with npm, suddenly it was trivial to include other packages without having to worry about sub-dependencies. The whole tree just magically added itself to your build, and only if you paid attention to the build process would you discover just how many developers you were transitively trusting.
Programs in other languages with the same kind of tooling tend to stop at hundreds or at most low-thousands of dependencies. Javascript code often reach tens or low-hundreds of thousands of dependencies.
Dependency explosion is bad all around, but JS is exceptionally bad.
https://github.com/rust-lang/docs.rs
Follow the build instructions, then you get to `cargo build` you'll see this message
Downloaded 448 crates (44.1 MB)
react@15 — 20 deps
react@16 — 6 deps
react@17 — 4 deps
react@18 — 3 deps
If you pick your packages carefully (for example, looking for alternatives this one ↓ instead of the much more popular react-router), it's not as bad as it used to be.