That will still allow execution of kernel code you may not want executed (even though in a VM), and is thus strictly worse. Especially for the modem, which you need to run and interact with all the time on a phone, this sounds like overkill.
It's better to not expose the needless attack surface rather than just attempting to isolate the attack into a leaky VM.
What is the alternative? If you don’t use a VM you have to expose dom0 directly to device io.
EDIT: ah I see we’re talking about a Linux phone! I agree with your point I’m not sure mobile devices have the resources to isolate every component into a VM. Overkill indeed!
I’ve looked a few times for an ARM SBC with a nice architecture to play with this stuff, Nvidia seemed to be the only option but they are also unobtanium right now.