Can you explain more about this feature? If it is such an improvement, why would it not be enabled by default - are there any drawbacks to enabling it?
* https://git-scm.com/docs/git-update-index#_untracked_cache
The underlying filesystem needs to have certain features enabled (mtime of directories) to be able to support the untracked cache. There's no guarantee that all will, so it does not enable it.
I suppose this could be enhanced to be an opportunistic feature. However, then there is some variability in what git is doing versus there being one default behavior.
The reason it's disabled by default is first of all that it's relatively new, but secondly that if your FS stops behaving as expected (or you move your repo to one that doesn't behave) it'll degrade very badly, i.e. "git status" might now completely miss files that have been modified in your working tree.
But if you're just running a system where you know you can trust the FS you can use the untracked cache and get a lot of "git status" speed-up, which'll matter more the larger your checkout size is in terms of checked out directories & files.
% ~/g/git/git-update-index --test-untracked-cache
Testing mtime in '/media/psf/Home/Downloads/junk' .
directory stat info does not change after adding a new fileIt would be a pity if an ignorant user copied a repository to another filesystem only to find out that certain operations suddenly break.
Perhaps the option should include the filesystem UUID for which the option should apply (to prevent these kinds of surprises).
it can by using `git update-index --test-untracked-cache` but it takes a long time to check this, so I guess they don't want to incur that penalty on every run.
OTOH,
>It would be a pity if an ignorant user copied a repository to another filesystem only to find out that certain operations suddenly break.
yes. I agree. The way this is handled at the moment is dangerous to the point of this not being worth it at all.
How about just writing a file to a scratch directory, and see if the mtime of the directory was changed?
The check you're talking about is one of the many checks it does, but there was further breakage which all needs to be checked against.
If so, they have to check every directory in the tree (or, alternatively, figure out all mount points in a different way, but I don't know whether that can be done efficiently in a portable way)
Once tested (or enabled without testing if you know what you're doing), the location of the repository and uname are recorded. If you copy the repository elsewhere, git should detect and disable the cache.
1) the gain is too small to be useful. Its still too slow for a synchronous poll in an IDE/editor or something so anyone concerned about latency will spawn a thread or process to asynchronously check the status. Now if we were going from hundreds of ms to tens of us... but we aren't.
2) the gain can be expressed in years of hardware evolution "just wait X years and your faster mass storage will naturally speed up enough...". vs the time it would take to implement bulletproof code. Better off just waiting for faster drives. Simple code on fast drives beats complex code on slow drives.
3) speaking of faster drives, $$$ can be turned into speed in a pretty smooth interchangeable market of professionals and exotic hardware. This is extremely well developed and widely understood. Pop in a SSD, parallel off the NAS, whatever. Practically nobody knows how to troubleshoot the git enhancement. Best of luck, as an overall lifetime system cost on a large scale its going to be an extremely expensive way to gain performance compared to slapping in a SSD.
4) The larger the code is the more complex it is and the fewer people can understand it. Industrial revolution dogma about specialization doesn't work in code. So outside the new feature, the rest of the code will suffer because the bar will raise that much. Simplicate and add lightness.
Some of the technical reason workarounds are missing "fun" scenarios like some madman has a git repo spanning multiple filesystems so you can't just check the repo root directories capabilities you have to traverse the whole repo tree (ugh) or there's a backup-restore cycle where the filesystem type changes (maybe as part of a hardware and OS upgrade?) or there's layered filesystem problems (stored on ext3, but exported via an obscure userfs or networked file system). Also caching must be hilarious, doesn't NFS have mount option timers like acdirmin acdirmax and noac that mess with when mtime is updated, so two clients on the same NFS mounted dir could react differently based on wall time the command is run, LOL that one would be fun/hilarious to troubleshoot.
1) I find that for the large repos I work on the difference goes from "noticeably slow" to "I don't notice it" which is somewhere on the magical ~200ms boundary.
2) This really doesn't help, the Git repo is likely in the FS cache anyway, you'll get exactly the same results / speedup if you do this in /dev/shm i.e. the in-memory filesystem.
This has nothing to do with fast drives, it has to do with syscall overhead. Recursively stat-ing a huge directory is simply never going to be all that fast.
3) Irrelevant for the reasons noted above.