Hacker News Reader

Comment by donatj | Hacker News Reader

Excuse my ignorance, but couldn’t they just add a SHA256 hash to commit objects (or some new commit-verify object) of the entire trees current concatenated content, leave everything else SHA1 and get the same benefit without rewriting the entire thing from the ground up? Git could even do that as part of the git gc step slowly over time - tag commits with a secondary hash.

Rewriting the whole thing including every git repos history seems like throwing the baby out with the bathwater, when you could just add a secondary transparent verification instead. Just seems like there has to be a better way.

View on HN

You can't change past commits to add that hash (without changing all commit hashes), so this method could only protect new commits. For any existing repo this would lead to a very weird security model: We admit that sha1 hashes are broken, and only guarantee that commits made by git versions newer than git x.x.x are safe from after-the-fact modification (or alternatively only commits made after date X).

My inclination is that protecting only new commits might be enough, but it gets me thinking: What would a practical attack on this look like, assuming sha1 was broken? Let's say I'm trying to insert a line of code that does something nefarious, and that it's now trivial to generate "magic text" you can stick anywhere in a file (eg, inside a comment at the end of a line) to get any desired sha1 hash.

Are all the other future commits still valid, or am I going to suddenly get conflicts or garbled text? Depending on where the modification is done, that code might have gone through much more churn -- especially if there are a bunch of sha-256 commits after it (which I can't attack). I don't know enough about how git stores content blobs to answer this.

Second problem: Can I push my replacement commit to another repository (eg, github)? Would even force push work? Do I have to delete branches and re-push my own? If I already have enough permission on the repository to do this, it means I can already push whatever I want -- so does this attack even matter at all?

Assuming that's successful (or I can trick people into using my own repository), what will happen to someone that already has a clone and does a pull? Will they get my change (and will it work or be a pile of conflicts or garbled text)?

Even if only fresh clones will get the changes it could still be quite devastating -- especially if using CI -- but I'm just not clear if this attack is even theoretically possible.

> My inclination is that protecting only new commits might be enough

Why? It's not the same as saying 'versions after vX are safe', it's the same as saying 'any unsafety after vX was there before, not introduced since' (both with 'as a result of SHA-1 collision' qualifiers of course).

> Can I push my replacement commit to another repository (eg, github)? Would even force push work?

Implementation dependent I suppose, but I wouldn't have thought so - I don't see why they'd actually check the content when the hash is supposed to indicate whether it differs or not.

> Do I have to delete branches and re-push my own? If I already have enough permission on the repository to do this, it means I can already push whatever I want -- so does this attack even matter at all?

I think an attack would look more like:

  1. Create hostile commit that collides with extant commit SHA
  2. Infiltrate a package repository, or GitHub, or corporate network, or ...
  3. Insert hostile commit in place of real one

Of course it's a problem if 2 & 3 happen alone anyway, but the problem with the collision commit is that it makes it so much less detectable.

Git commits are snapshots, not diffs. Each commit contains a tree, which contains a list of files and their respective hashes. As long as its whole tree is SHA-256 then a commit should be safe, regardless of its history.

The downside to the migration would be that all unchanged files would be stored twice (once identified by SHA1, once identified by SHA-256). But you could work around that by hardlinking identical files.

This doesn't protect subdirectories unless you rewrite the entire tree structure with SHA256. I don't know if Git does that now, or not. Git generally points to unmodified subdirectories with the existing content hash; if the SHA1 is pointed to by SHA256, which is implied by the transition plan proposed in the grand-grandparent comment, then those subdirectories are essentially unprotected.

> Are all the other future commits still valid, or am I going to suddenly get conflicts or garbled text? Depending on where the modification is done, that code might have gone through much more churn -- especially if there are a bunch of sha-256 commits after it (which I can't attack). I don't know enough about how git stores content blobs to answer this.

A blob is a "snapshot" of a file. The next version of a file is a completely different blob with no direct relation to the previous.

"Pack files" use delta compression in order to lower the actual size of "similar" blobs.

You could get conflicts if you tried merging or rebasing over the nefarious blob, and the "patch history" (git log -p, which builds the patch view on the fly) would show possibly unexpected complete file replacements.

Couldn't they make a table that contains a list of all the old objects by SHA1 hash, for for each contains the new SHA256 hash of that object, and then commit this table in the repository?

Yep.

I’m not known with the internal data structure of git, but couldn’t you add the new hash as a commit in a new format “on the side”, leaving the original commit as is?

This is kind of what rewriting the repo is. Yes, you could leave the SHA1 commit tree around afterwards (i.e., for convenience of existing URLs), but you wouldn't want to keep SHA1 around as the authoritative hashname.

Git does have an commit-related object called a note that you can attach as a separate object. [1]

Presumably the proposed "hash translation store" could use an approach similar to notes, and include the hash translations as objects in the git database (hopefully in a way that could be signed by a tag).

[1] http://alblue.bandlem.com/2011/11/git-tip-of-week-git-notes....

What if we use the exploit to add the new data but keep the sha1 the same? :)

Hashing everything in one go doesn't scale well. When making a new commit you want to only hash a proportional part of the repository, and the tree structure of git allows that, only the files and "tree" objects (directory listings) that change are hashed again.

Reply on news.ycombinator.com