Enigmail for Thunderbird: http://www.enigmail.net/home/index.php
GPG4Win provide plugins for Outlook: http://gpg4win.org
Evolution on Linux has OpenPGP support built in.
FYI: Another term used for OpenPGP is gnupg or GPG, should you want to google for other solutions on other operating systems.
But to be clear, this is just an interface to the MacGPG backend, so it's not some proprietary format. Your friends, etc. don't need to use GPGMail in particular; any PGP implementation will be fine.
With asymmetricity, the public key is a key which can only encrypt the message, but even the sender cannot decrypt that same message again with that key. Only the single unshared private key can decrypt them.
This ofcourse means that all parties must have their own key pair, and the public keys have been shared between them. Also they must use a GPG compliant program to encrypt/decrypt or sign and verify the messages.