Then they're effectively "leaving money on the table" by avoiding female founders, because they're being too risk-averse. The risk of false accusations that will blow up are too small to worry about.
If they're not outright bad investors, they are at least investing a suboptimal way.