The solution the supply chain problems is removing your deps from .gitignore | Hacker News Reader

The solution the supply chain problems is removing your deps from .gitignore

10 pointsSchizoDuckie12 comments

I head the major realisation today that we've all been bamboozled.

All the supply chain attacks currently happening would never even happen if we just checked in our language-respective vendor/node_modules/venv directories into git and just deployed straight from that.

Screw the dependency install and upgrade step. Screw the automated build step. Screw the breaking changes because $package_owner doesnt adhere to semver.

Checking in dependencies and their updates individually is, and has always been the way out of this mess.

Remove vendor/ node_modules/ and venv/ from your .gitignore today and skip the install step in your CI and you eliminate 99% of the attack surface instantly. Was it always that easy???? I think it was!

You think checking in your composer.lock or package.lock saves you? Hah. Npm install is "smart" and checks for updates and silently installs new versions and updates your lockfile. You should have used npm ci instead. We actively train devs to run 'composer update' to check for new releases that fix 'issues' they might encounter locally and delete the lockfiles as a first measure to fix issues.

Do you vet every update to your composer.lock? That one innocent commit hash that's changed could just pull in 20kb of obfuscated exploit code and you'd never know.

All of this is compounded by the longstanding hilarious github bug where you can fork a repository and push your commit to it, then pluck the commit hash and append that to the original repository URL. On the Github webinterface you'll see a notice "this commit has might not belong to this repo or a fork of it" but on the terminal you'll never see that, and that's exactly what the current worms exploit.

Checking in your dependencies and eliminating the install step would make all of this trackable and traceable. Imo the performance hit is worth it.

I'm glad other people are starting to realise this.

Just a word of warning from someone who's always advocated for this approach:

Prepare to be inundated with ridiculous, nonsensical arguments about why it's impossible to work that way.

It totally works, but only if you manually review every change in every dependency and transitive dependency you rely on.

If you blindly copy in changes how is it any different from npm pulling in those same changes, other than being much slower?

Being much slower is the mitigation.

I mean even if you blindly copy in the dependency (I'm not saying you should) you've already solved two supply chain issues with install-on-prod:

- dependencies "disappearing" (aka left pad 2, electric boogaloo)

- dependencies running nefarious "install scripts" on prod

Apparently some language package managers also will silently install newer versions than a lock file specifies, if you use the wrong install command. So that's arguably more a case of saving you from yourself but the example I saw said that "... install" is wrong you need to use "... ci" which is kind of asinine IMO.

Things like sudden changes in dependencies should also be noticed more readily.

Copy/pasting source code doesn't solve either of those problems either.

If you already had left pad cached then you were not affected by its disappearance.

If a package needs an install script to be used, to compile some native code for example, you still need to run the install script before you can use the package.

Manually repeating the actions npm does automatically does nothing to protect you from supply chain attacks.

The only thing that helps is to review code before you run it. How the code got to your machine before it was run does not matter.

The real vulnerability isn't the supply chain, it's the muscle memory of running install and update reflexively without reading what changed.

> If you already had left pad cached then you were not affected by its disappearance.

The evidence of course is that when you say "left pad" no one knows what you're referring to because nothing bad ever happened.

> If a package needs an install script to be used, to compile some native code for example, you still need to run the install script before you can use the package.

This already sounds like a giant red fucking flag, but sure whatever, what you're using needs some compile step. You can be in control of what runs when and where. Heck you could even take the fucking maverick solution and compile the shit once out of band and deploy the compiled binary to your production environment. I know that forethought and planning ahead will come as a fucking shock to the NPM using community, but try it some time, it's really kinda good.

> Manually repeating the actions npm does automatically does nothing to protect you from supply chain attacks.

I mean it does, inherently, if you're running those actions locally in an environment without production access...

Oh noes, your compromised module stole your DB credentials, and your SES credentials to spam all your customers..... and just got a bunch of failures or sent messages to no-one because the environment has dummy data and uses a locked down sending configuration for SES.

There is 100% benefit in running that shit in a development environment.

> The only thing that helps is to review code before you run it.

Right, and somehow you think having the code in question literally in your VCS, waiting for a merge like all the other code... that's not apparently helpful.

But hey thanks for proving me right about the unhinged complaints. Stong echoes of "we've tried nothing and we're all out of ideas".

> There is 100% benefit in running that shit in a development environment.

Most of the recent supply chain attacks specifically target stealing secrets from development environments.

> But hey thanks for proving me right about the unhinged complaints.

I'm sorry if I'm upsetting you, but I am not complaining or trying to provoke you.

"Just check in the dependencies and review them" is not a revolutionary idea. It makes sense in many contexts.

But it is not practical in the overwhelming majority of contexts. React's last minor version bump included 100 files and ~5k changes. It also bumped the versions of 6 direct dependencies, which in turn bumped dependencies, etc.

It is not reasonable for a small (or even medium) team using react to manually review all of the changes and all the changes in react's transitive deps each time they need to update. The problem grows exponentially with all of the other common projects and libraries likely being used in a front end project(vite, react router, redux, vitest, etc).

pick a few non-trivial npm projects and try to audit all of the changes (for all transitive dependencies) for a few releases that bumped dependencies.

> Most of the recent supply chain attacks specifically target stealing secrets from development environments.

If your secrets in a dev environment can actually do any damage if leaked, you're doing something very fucking wrong.

> React's last minor version bump included 100 files and ~5k changes.

So you're choosing over engineered dependencies and then complaining they're too big.

Somehow I think the problem, as usual, started with the meat sack on the chair.

I think we may be talking past each other at this point, but secrets from a dev env can be more valuable than secrets from a production env.

With things necessary for a dev env, like read/write access to source control, attackers can get access to internal data, and push malicious code that gets run in a prod env anyway.

If you want to make the claim that using react is an insane indefensible choice from a security standpoint, you are being idealistic at best.

Telling people not to use react does not help anyone, and that type of recommendation causes reputational damage to the security industry.

We are clearly talking about different things.

Access to source control is required on a developer workstation.

It is not required inside an application environment on that workstation (eg a VM or other such system that both provides a standard environment and creates separation)

I'm not making any claims about the security of react.

What if you deploy to a different CPU architecture than you develop with? Some languages/libraries use pre-compiled binaries as part of their packages.

Reply on news.ycombinator.com