Code signing is used to prevent this. So, either the attacker has an Apple developer account (and is hopefully traceable through their credit card information), the Transmission project was sloppy with their signing key, or the machine of the developer with the signing key was compromised.
People have been ranting negatively about the Mac App store. But this is exactly why we need sandboxed applications by default (which is what the Mac App Store enforces). A sandboxed application cannot take your data hostage.
(Yes, I understand that App Store distribution is probably not possible for a Bittorrent Client.)