That seems like a naive take. If any of your local VMs are internet connected and are compromised, side channel attacks could be used to exfiltrate data from other VMs or the host.