Curious about the details behind those compatibility problems.
Curious about the details behind those compatibility problems.
The whole ARC thing for example, sidestepping the general block cache, feels like a major hack resulting from how it was brutally extracted from Solaris at the time...
The way zfs just doesn't "fit" was why I had hope for btrfs... ZFS is still great for a file server, but wouldn't use it on a general purpose machine.
For instance using the `zfs` tool one wouldn't only configure file system properties, but also control NFS exports, which traditionally was done using /etc/exports.
ZFS and ZPOOL tools provide accesses to multiple different subsystems in ways that make more sense to end user, a lot like LVM and LUKS do on top of device mapper these days
ZFS sidestepping conventional device and mount handling with the way it "imports"/"exports" ZFS pools, usually auto-mounting all datasets straight to your system root by default - if you have a dataset named "tank", it automounts to "/tank".
ZFS operation itself being an inexact science of thousands of per-dataset flags and tunables (again not set through the common path with mount flags), and unless you run something like TrueNAS that sets them for you it's probably best to pretend it's not there.
Common configuration with decent performance commonly involving complexities like L2ARC.
It's far too invasive and resource consuming for a general purpose machine, and does not provide notable benefit there. A dedicated file server won't care about the invasiveness and will benefit from the extra controls.
btrfs fell short by still not having its striped RAID game under control, only being "production grade" on single disks and mirrors for the longest time - probably still?
One huge problem with ZFS is that there is no zero copy due to the ARC wart. Eg, if you're doing sendfile() from a ZFS filesystem, every byte you send is copied into a network buffer. But if you're doing sendfile from a UFS filesystem, the pages are just loaned to the network.
This means that on the Netflix Open Connect CDN, where we serve close to the hardware limits of the system, we simply cannot use ZFS for video data due to ZFS basically doubling the memory bandwidth requirements. Switching from UFS to ZFS would essentially cut the maximum performance of our servers in half.
Better to just have the filesystem get out of the way and just focus on being good at raw I/O scheduling.
I wonder if FreeBSD is going to get something io_uring-esque. That's one of the more interesting developments in kernel space...
FreeBSD has "fire and forget" behavior in the context of several common hot paths, so the need for io_uring is less urgent. Eg, sendfile is "fire and forget" from the application's perspective. If data is not resident, the network buffers are pre-allocated and staged on the socket buffer. When the io completes, the disk interrupt handler then flips the pre-staged buffers (whose pages now contain valid data) to "ready" and pokes the TCP state machine.
Similarly, FreeBSD has OpenBSD inspired splice which is fire-and-forget once 2 sockets are spliced.
But to be fair, we are approaching the point where spinning rust stops making sense for even the remaining use cases, and so designing new optimizations specifically for it might be a bit silly now.