If something can be done from userspace, it should be done from userspace.
Legacy Linux drivers could be sandboxed in user space inside a fake Linux kernel environment, a bit like how captive NDIS used to provide a fake NT kernel for network drivers run under Linux.
The fact that the kernel ABI is unstable is irrelevant because you're not relying on ABI stability. You're using a specific version of the core kennel paired with a specific version of the driver.
Yeah your kernel could totally support USB coffee makers using USB passthrough to a Linux VM as long as (1) you write a Linux daemon for each device that lets you talk to the hardware over RPCs from the host and (2) all application software that wants to talk to said hardware uses your dbus API. That's a hell of an ask, but it's doable.
But what about the xHCI drivers for USB? You still have to write that for every USB chip on the market.
What about file systems? I've been burned by FUSE performance.
What about GPU drivers? This sucks on Linux and it'll suck even harder on your hypothetical kernel.
Or any device that sits on the PCIe bus for that matter. There are so many drivers that rely on running in a shared address space with data structures and hardware state shared with the rest of the kernel. At the very least you'll have to support the 2 new motherboard chipsets that come out every year. To be clear, I'm just taking about desktops. Laptops and phones will be a whole new world of pain