Hacker News Reader

Comment by Borg3 | Hacker News Reader

You are right. GIT is not DVFS, its DVCS. It was made to track source code, not binary data. If you are putting binary to DVCS, you are doing something wrong.

But, there are industries that need it, like game industry. So they should use tool that allow that. I heard that Plastic-SCM is pretty decent at it. Never used it so cant tell personally.

Replacing GIT is such a stupid idea. There is no ONE tool to handle all cases. Just use right one for your workflows. I, for example, have a need to version binary files. I know GIT handles them badly, but I really like the tool. Solution? I wrote my own simple DVFS tool for that usecase: dot.exe (138KB)

Its very simple DVFS for personal use, peer to peer syncing (local, TCP, SSH). Data and Metadata are SHA-1 checksumed. Its pretty speedy for my needs :) After weeks of use use I liked it so much, I added pack storage to handle text files and moved all my notes from SVN to DOT :)

View on HN

DVCS stands for distributed version control system, it has nothing to do with source code?

Maybe you're confusing it with SCM which are source control managers, that's the only ones that handle strict source only, but scm can mean other things.

Hard to say.. For me DVCS is more advanced version of DVFS. DVCS can do branching and merging, provides more metadata for revisions etc.. DVFS just do pretty much one thing, store binary blobs. And because binary blobs cannot be easly merged, I would not use it for storage here. But I guess, its just me :)

What is this dvfs you talk about? desktop virtual file system? Dynamic voltage and frequency scaling? Google doesn't know, or well, the first page doesnt know.

DVCS stands for distributed version control system, it has nothing to do with binary blobs, whether or not it supports it, is an implementation detail.

VFS = Versioned File System; D means Distributed.

Isn't there git-annex [0] if you want to store large binary files?

[0] https://git-annex.branchable.com/

Yeah I know about git-annex. It might be good solution for big data. In my case, I do NOT want to decouple storage from metadata. I want single repo for single project that is self-contained. Easier to manage, its truly distributed. No need to bother w/ backups because every replica have everything allready. Its good model for several GBs of data.

Second that way of thinking, for me GIT is as good as it gets for versioning text files.

Not handling binary files is not a downside for me because GIT should not be a tool to handle binary files versioning and we should use something else for that.

What do you use when you have a tiny .png or .jpg that needs to live alongside your source code now?

Just put it in your git repo.

I can put a binary file in GIT repo especially small ones and ones that don't change - things that people want are "handling binary files well", whatever that means, but putting big binaries in GIT or a lot of binary files or versioning them is not the use case for GIT.

I think what they mean by "handling binary files well" is being able to selectively not clone/pull them. Otherwise, binary files are handle as any other files, except you are not shown a diff of them by default.

You are wrong.. Just start putting files of GBs in size and you will see what happens.

So I just added a 2.4GB iso and other than taking longer nothing happened. Push and clone also seem to work.

Right :) I forgot that everyone is in 64bit systems these days. This change a lot. Anyway, GIT uses single blob for single file content. This is all right, if we speak about source code and text files, as those are not large. Thing changes for arbitrary binary files.

AFAIK, not much of anything changes (perhaps name some of these changes?).

And what is wrong with a single blob? This seems work great for deduplication, if you were using git to manage a photo library and happen to add something twice.

There is several issues depending where you run git on. For example mmap() can be problematic on large files in 32bit OS. On Cygwin it gets even worse.

As for deduplication, I do not think so. If you have single blob, lets say 1GB and you change just 1B, whole blob changes and no more dedup. If you use basic method of static block size, lets say 512KB, this will work much better. Futher, there are more advanced techniques to handle dedups like roling checksums to carve out even smaller sub-block.

But you're always arguing the odd case: - 32 bit systems - cygwin git on Windows instead of native - expecting deduplication on different files (???)

Uh, hold on. Are we trying to discuss objectivily? If not, and you just insist like GIT is best for everything fanboyism, and use it everywhere.. I am not interested.

I pointed out that GIT have some rough cases and its good to be aware of them. I said it already and will say it again: choose right tool for the task.

If your workflow with GIT works, use it. No need to discourage other solutions. Yes, I use ancient platforms and I want DVFS on them too.

Well, you said something will happen with git and big files, but it seems not much will happen for the majority of users.

Reply on news.ycombinator.com