I've used Firefox since the very earliest betas and it has never, ever been exploited to infect my machine. Sure, it's had big holes, but I've never seen them widely deployed in the wild. I ran without antivirus software from 2003, because it just wasn't necessary.
Chrome, on the other hand, has been clearly and unambiguously the only possible vector for three separate, massive viral infections on my machine in the last 6 months, infections severe enough to take out my entire OS and require a reinstall[1]. Sure, in two cases they were viruses delivered by ad networks on sites that I probably shouldn't have been visiting -- but one was just a rogue ad hit while browsing Twitpic. Now I run Antivirus software again, and every couple of weeks a compromised ad network -- even on quite reputable sites -- will trigger an alert and automatic cleaning. For some sites I switch back to Firefox.
Chrome's amazing performance has to come from somewhere, and my impression is that that boost comes because it runs "closer to the metal", i.e. fewer layers of virtual machines and sandboxes. But that design decision has drawbacks, and one of them seems to be security.
[1] Infection details: Chrome running on Windows XP (first two times) and Windows 7 (final time). I didn't do forensics on the actual vector, but all three times rogue Java applications started up and then installed additional infections, which appeared within seconds of the first attack, making it hard to say which virus was the first to break in.