Most fees go to the issuing bank. The issuing bank is usually on the hook for customer disputes. So I think you are mistaken.
https://usa.visa.com/content/dam/VCOM/global/support-legal/d...
> If the dispute is valid, the acquirer deducts the amount of the dispute from the merchant account and informs the merchant.
Also, keep in mind that companies like Stripe will not only recover the disputed amount from the merchant but will also recover a fee from the merchant!