Hacker News Reader

Comment by mgaunard | Hacker News Reader

if you have large files in your repository, you have a design problem.

Not in gamedev where you can have hundreds of gigs of art assets (models, textures, audio...), but you still want to version them or even have people working on them at the same time (maps...). But that is a different can of worms entirely.

Indeed I have 3D assets in this case. Would this be done differently in an enterprise that has all kinds of tools to manage specialty workflows? Sure. Do I want to spend my days configuring and maintaining some binary blob / LFS storage system? No.

I’ve migrated a lot of projects from fossil to git eventually, but I dare say they never would have made it that far, had I started out with more friction, including fighting vcs tools.

git is for when you want to track changes to part of a file.

In your scenario, you just want to track different versions of a file.

You can equally say that git is for when you want to track changes. And then it's a failing of git.

Besides, what's the difference? It's a file. The contents changed. Git doesn't say anything at all along the lines of "30% or more different means it's not a good fit for git".

You can certainly use git, but then the model used to apply patches from different branches doesn't work for binary blobs.

So all the things that git specializes in are actually things you don't want.

That seems like an implementation detail that could change tomorrow, at which point it could be perfectly fine to store large blobs in your repository, yea?

I completely agree Git is bad at this now, to be clear. I've watched single-file repositories bloat to hundreds of gigabytes due to lots of commits to a single 1MB file. But that doesn't seem like a design problem, just implementation.

Then use a binary repository like Artifactory which has LFS support. You can still version them in git- just don’t store them in there

> Then use a binary repository [...] You can still version them in git- just don’t store them in there

So git works, as long as you include things that aren't git to handle what it can't.

Stockholm syndrome really blinds people

Not sure if you were agreeing with me or not BUT I run into this often where people do not use the right tools and try to make one tool fit every CM scenario. SharePoint sucks but it has its place. Along with Artifactory/Nexus

That's what object storage with versioning turned on is for e.g. GCS or S3

Although blob storage work well for versioning, you have to make heavy use of the underlying proprietary API to get these versions, and I am not quite sure you can do more complex operations, like diff and bisect between those versions the way you could with git.

Why use git at all then? Just use an object store with versioning turned on.

Because git excels in relatively small size text files and patching and difficult. You can't binary blobs like jpegs, audio, video easily.

But that's my point: why can't a version control system be good for this as well? It's the same thing underneath. Why do we have to split these different use cases across different tools and hope a foreign key constraint holds?

We've got lots of disk-backup tools that handle this just fine, deduplicating blocks and compressing where they're able.

That's a ridiculous claim. Can you really not think of a single situation in which it makes sense to keep track of big pieces of data alongside (or even instead of) source code? The fact that many VCS don't handle large binary data nicely doesn't mean there's never a good reason to do so.

> Can you really not think of a single situation

It doesn't even matter if they can think of one; assuming your own use cases for software are everyone's is proceeding from false premises and is the sort of thing that leads to (and necessitates) "hacky workarounds" and eventually the adoption of better software we should've had in the first place.

Assume nothing about user's use cases. A VCS should not be imposing arbitrary limitations on the files it's indexing. It's like the old-school filesystems we (surprise, surprise) deprecated.

> if you have large files in your repository, you have a design problem

Your workflow and use cases aren't everyone's.

A workflow design problem?

Reply on news.ycombinator.com