The vendor already eats that risk though, right? Chargeback fees cover the credit card company against that risk.
The problem from the credit card companies' point of view is that "I got caught by my wife" isn't a valid chargeback reason, so the vendor can dispute the chargeback, have every incentive to do so, and that's not a fight Visa want to get into.