Hacker News Reader

Comment by dilap | Hacker News Reader

Very cool stuff! I think built-in excellent support for large files & large repos could be a huge driver of adoption vs git (which, of course, has poor support for those things).

(At work we use git, but split into many repos, which is a pain, and with git-lfs, which is also a (mild) pain.)

View on HN

Git doesn't have bad support for large files. It just doesn't make special exceptions for not recording their history like Git LFS does.

I don't see anywhere where Pijul would be different to be honest.

True, you can put large files in git, but after a while your .git will become huge, clones will become extremely slow, and (if you're hosted on github), github will threaten to delete your repo. ;-)

Then you have to use something like git lfs, which isn't terrible, but also isn't 100% smooth either.

Something that handled this as part of the native experience out of the box would be much nicer.

Edit: Pijul would be different by not having clone become slow, as I understand the blog post.

Sure but all of those things are a consequence of Git doing its job of recording all history forever. Git LFS is only "better" because it lets you delete history later and it enforces getting shallow clones.

Maybe Pijul solves that magically somehow (e.g. can you change history without changing commit hashes?) but they haven't explained how IMO.

Well, I can just report my experience with a git repo w/ a bunch of binary files in it.

It got really really slow to clone (like multiple days -- we gave up on cloning and just copied the checked-out folder around); git lfs fixed this. I should note most of the binary assets had no changes and were not deleted; it seemed to be something inherent in git's handling of the binaries that made it slow, not just downloading the data. (I.e., I don't believe most of the wins in git-lfs were simply in doing a "shallow clone" as it were.)

So as I see it the situation is as follows:

- Even for a full clone, git becomes inefficient with a lot of large binaries

- For doing a partial clone, things like git-lfs work, but are clunky because git-lfs is not natively integrated with git

- The partial clone abilities of git-lfs are strictly "time-based" -- you go to a particular checkout, and then it goes off and downloads the files. There's no way to say, "I have a big monorepo with a bunch of binaries, but I'm only interested in some subset w/o the binaries."

So these are all areas where I think git can be improved on.

That sounds like a bug to be honest. If the files never changed then you'll only have one copy of them and Git really shouldn't have to do any extra work to deal with them.

Certainly possible. & of course git is always improving. But still, I do think ability to do path-based partial checkouts and have something like git-lfs integrated natively remain compelling potential features.

I saw some mentioned ways to represent binary patches, but wonder if there is there also some sort of chunk-level deduplication?

Not yet, but this is just by lack of time for implementing the diff algorithm. There's nothing hard there, but there are other priorities at the moment.

Reply on news.ycombinator.com