Beware that VMs are not necessarily secure. They can be escaped!
VMs have tons of well documented issues. If you want a smaller attack surface, try OS virtualization technologies (zones/jails)
The way I see it, they're more secure that running the same apps on bare metal. Ubuntu host running a Fedora VM; the latter (with Transmission etc) only running when I need the apps running - seems an almost entirely painless way of providing a lot of security.
* http://arstechnica.com/security/2015/05/extremely-serious-vi...
* http://www.securityfocus.com/archive/1/495095
* https://www.cert.be/advisories/oracle-vm-virtualbox-two-priv...
* http://www.coresecurity.com/content/virtualbox-privilege-esc...
Lets try to avoid treating security as something binary.