The only parties to a trade are the buyer and the seller.

If both don't think they'll be better off doing the trade, it won't happen. No one else is materially affected.

You can think of this as how trade creates wealth!