Which results in a corrupted repository, from Git's point of view.
I think many modern file systems try really hard to make that true, but I don’t think you can count on it. “man write” (https://man7.org/linux/man-pages/man2/write.2.html) still says:
“Note that a successful write() may transfer fewer than count bytes”
It also says
“A successful return from write() does not make any guarantee that data has been committed to disk. On some filesystems, including NFS, it does not even guarantee that space has successfully been reserved for the data”
That should be handled by calling fsync, but of course, if that fails, there’s not a lot you can do (even if you exactly know what happened) (https://research.cs.wisc.edu/adsl/Publications/atc20-cuttlef...)
I also don’t think calling data loss due to writes that do not make it to disk “file system corruption” is correct. For file system corruption, the file system data structures have to be overwritten (e.g. the boot record or directory data structures)
Not sure what you mean by entire fs transaction. But Linux doesn't have transactional fs interface, so open, write, close can be interrupted at any point with result of just having a new empty file after open, being one of the valid outcomes after crash.
The same happens if you have a power loss or kernel crash on Linux (as a kernel developer, it happened to me several times when testing freshly-committed code).
I'd say WSL2 team, because it worked well in the original WSL.
FWIW I am a user moderator down there and VMware desktop products (Workstation/Fusion/Player) has my focus and as a result I basically read almost every post that would report this issue.
I just DDG'ed it and see one report from 2014 [0], ok some more from around 2008 when using google.
Looks like this was resolved in 2015 as that was the last time I see it being mentioned.
Took quite some time to figure out what was causing it. "Everyone is somehow corrupting files except for me, wtf?" Magic filesystem translation layers always suck
I can't recall what made me start doing that, but now it's a habit.
Same thing, sorta but all in one go.
So, to me, it looks like WSL2 not completely flushing writes to the underlying file system. Bad, but not as bad as file system corruption (which could lead to losing all data on the disk)
Also over the years it seems like everyone I've seen that habitually edits files remotely ends up with this sort of pain and butthurt.
The first instance of a problem is:
[ 1.956835] JBD2: Invalid checksum recovering block 97441 in log
And that's corruption that leads to log replay failing, i.e. rejecting it because honoring the replay in the face of checksum errors could make things much worse. Subsequently mount fails: [ 21.151232] ERROR: MountExt4:1659: mount(/dev/sdb) failed 5
That's good because the purpose of journal replay is to make the file system consistent following a crash/power fail. And if the file system is dirty, replay is called for, but can't happen due to a corrupt journal, so now an fsck is required. i.e. it is in an inconsistent (you could say partly broken) state and needs repair.I haven't seen syslog/systemd journal for other cases to know if there's instances of ext4 log replay that succeeds, but with missing files. That's not file system corruption, even if it leads to an inconsistent state in a git repository (or even a database). But this is still concerning, because to get a situation where log replay is clean but files are missing suggests an entire transaction was just dropped. It never made it to stable media, and even the metadata was not partially written to the ext4 journal.
qemu-kvm has a (host) cache setting called "unsafe". Default is typically "none" or "write back". The unsafe mode can result in file system corruption if the host crashes or has a power failure. The guest's IO is faster with this mode, but the write ordering expected by the file system is not guaranteed if the host crashes. i.e. writes can hit stable media out of order. If the guest crashes, my experience has been that things are fine - subsequent log replay (in the guest) is successful, because the guest writes that made it to the host cache do make it to stable media by the same the guest reboots. The out of order writes don't matter... unless the host crashes, and then it's a big problem. The other qemu cache modes have rather different flush/fua policies that can still keep a guest file system consistent following a host crash. But they are slower.
So it makes me suspicious that for performance reasons, WSL2 might be using a possibly volatile host side caching policy. Merely for additional data point, it might be interesting to try to reproduce this problem using e.g. Btrfs for the guest file system. If write order is honored and flushed to stable media appropriate for default out of the box configuration of a VM, I'd expect Btrfs never complains, but might drop up to 30s of writes. But if there's out of order writes making it to stable media, Btrfs will also complain, I'd expect transid errors which are also a hallmark of drive firmware not consistently honoring flush/fua and then you get a badly timed crash. (And similar for ZFS for that matter - nothing is impervious to having its write order expectations blown up.)
Silent corruption of parts of the disk that you rarely access but still want to keep is scarier (you might have rotating backups for months or years and still eventually lose data)
To avoid silent corruption requires full metadata and data checksumming, ala Btrfs or ZFS. In those cases, not only is corruption unambiguously detected, but it's not allowed to propagate.
Source: https://en.wikipedia.org/wiki/VMware_Workstation#Version_his...
To be fair, this feature always felt..rickety. But it was very nice.
You would be missing the remote power operations.
FWIW, I am working on a product called Vimarun [0] that is aimed to replace most of that missing functionality over time.
No remote power operations yet, but that will come.
I hadn't even considered that. Connecting USB devices remotely works well with vSphere, but never tried that with Workstation. They are going to completely removing the hostd engine from VMware Workstation. That would indeed also include that part and it is most likely not easy to get that working without hostd.