Yep this would be fantastic. The ironic thing about it is that a fine-grained write completion callback like you mention would actually improve performance compared to the coarse fsync we have today.

Another simpler way to improve the filesystem API would be to add a write barrier system call. If I call write(file, a); barrier(file); write(file, b); then the OS guarantees that the second write can't start before the first write is fully committed to disk. This is enough to implement most journaling systems, databases and log files.

This would perform much better than the current POSIX API because you don't need to "stop the world" with an expensive fsync before issuing subsequent write commands. The kernel can buffer chunks of writes separated by barriers, and issue them to the underlying device as needed.

The completion API is nicer though, because you can (for example), not respond to a network request to update the database until the changes are flushed to disk.