Free speech as in speech protected by the 1st amendment? That's very specifically meant to curtail the government's powers, not the powers of merchants who decline to do business with others.

You could make a pretty strong argument that overly stringent regulations that result in sex workers, etc., being unable to process payments are a violation of that, but I'm not aware of why Visa/Mastercard don't support Gab & Parler. If it's that, you have a point. Otherwise, not really.