Crash recovery for user-space block drivers
lwn.net
lwn.net
I've seen a couple of FUSE filesystems that don't implement a full filesystem, instead implementing the minimum required for a file to exist which itself can have any other filesystem created within it and loop-mounted. Notably a couple that use S3 as the backing store. This means you get to use a different storage mechanism without losing the semantics of your preferred filesystem.
If you have a user case for this (for one it could simplify integrating a new storage service into your infrastructure) then there is a use case for ublk: it achieves the same but more efficiently.
But this allows any typical block device driver to run in userspace which means you no longer require certain version of kernel to support new features or new protocol, which is nice.
It's also useful in testing or prototyping filesystems, as you can make userspace proxy that can emulate any kind of error without much kernel knowledge. Sure, there are already some tools for that but it's easier to do in userspace.
It used to be my daily driver, but three moves across country and small living spaces pushed this storage box down in urgency; I regressed back to my old bad habits of haphazard disks and flash drives and usb "thumb" drives... all over...
Anyhow, tried to import the ZFS pool, crashed ZFS, caused a kernel panic. Dang.
Switched to a recent FreeBSD distribution, same crash.
I'm pulling together the various methods of recovery from folks who've seen this type of failure, and all will be well. (Or if not, I've got at least one good snapshot up on Backblaze.)
But user space for something like this sounds like the perfect fit. ZFS already subsumes functions from device driver to blocks and files manager, on up to volumes and snapshots-- the whole storage stack. Those user-facing aspects of storage and volume management could reasonably be taken as a user space process tree, where a crash wouldn't bring down your entire server.
Heck, to triage my problem, it's suggested to re-run the zpool import with other options. If I could iterate and converge on a operational set of recovery steps, without taking my entire server out and back up from cold boot, yeah that would be great!
Is the mental model for this to treat it like a device driver (where it's off by default and can be turned on under specific circumstances), or is this thought of as part of the default-available kernel API that every system will have access to from now on? (I guess FUSE itself is a little fuzzy in this respect)
The code is tiny compared to full implementation of any of the block drivers and allows them to be moved to userspace so it also potentially saves a lot of maintenance in the future, because it would be the first choice if you're implementing another distributed block file system.
> Is the mental model for this to treat it like a device driver (where it's off by default and can be turned on under specific circumstances), or is this thought of as part of the default-available kernel API that every system will have access to from now on? (I guess FUSE itself is a little fuzzy in this respect)
As a device driver that can be fully tested with no actual hardware, and much smaller area of attack than real driver.
I wonder how long it'll be before someone merges a cleaner set of APIs for user space drivers to go with this. Android has already built out their own parallel feature for this though I don't think it'd fly for the rest of the kernel given how closely integrated it is. Neat nonetheless :)
Currently, if there is say a better, more efficient way to do some common task ,say storage drivers, do, you can just go in and refactor all of that.
With common interface, you're doomed. You might implement the better way and give it in API, but you can't force anyone to use it, and you can't deprecate the old one.
You also have to keep the old behaviour *forever*, even for things that "do not break userspace" so it's even bigger maintenance burden.
And what if your API is wrong ? Still gotta keep it... and someone might still use it and trip on the wrongness
https://www.kernel.org/doc/html/latest/process/stable-api-no...