Or just use Qubes OS with a dedicated VM for usb devices: https://qubes-os.org.
It's better to not expose the needless attack surface rather than just attempting to isolate the attack into a leaky VM.
What is the alternative? If you don’t use a VM you have to expose dom0 directly to device io.
EDIT: ah I see we’re talking about a Linux phone! I agree with your point I’m not sure mobile devices have the resources to isolate every component into a VM. Overkill indeed!
I’ve looked a few times for an ARM SBC with a nice architecture to play with this stuff, Nvidia seemed to be the only option but they are also unobtanium right now.