Or you could just implement the linux syscalls directly in the kernel, but that seems a bit self-defeating if you want to explore new ideas.
Just seems like a good starting point than reworking the entire world.
Also, on the question of "how" you'd do this, I'm not a kernel hacker, but I thought it would be possible to expose FFI, i.e. the shim you talk about, that would allow drivers to be used from something like NetBSD. But you're right that it could be self-defeating in that supporting all of those interfaces might end up constraining the OS in it's attempt to deliver on these features.
On a side note, it seems like it would be nice to expose POSIX FFI hooks to make porting software to a Rust OS easier.
I dunno about performance impact, but I bet it is fairly negligible in the grand scheme of things. In the rump kernel case, these shims really don't do anything, they just pass the call through. Think of it as an API facade. In a hobby OS it might be heavier, if your OS doesn't have a 1:1 mapping of the POSIX/linux syscall to an equivalent.
Also, we virtualize this stuff all the time in cloud environments. Your java app uses the JVM to execute a linux syscall which the kernel translates into a Xen hypercall which Xen uses to call a native driver which talks to the disk, etc etc. You obviously do pay a performance penalty for stacking more and more layers, but it isn't as bad as you'd think (or as good as you hope, take your pick).
But the point is that layering abstractions like this is very much alive and tolerated today in moderately high performance systems :)