The charge is to the merchant, not to the customer. In general, the price marked on the good factors in the credit card fees, because almost everyone uses credit cards. So, if a buyer uses cash or a debit card, he should always negotiate for a 1-2% discount.

Edit: To clarify, some U.S. merchants do charge a "credit card fee" or enforce a minimum charge amount to all customers using credit cards for their purchases. I have never seen one charge a surcharge only to Visa users or only to Amex users.