Run ssh-add in your terminal session before doing your push/pull dance — this way you only have to enter the password once. This gives you the security of the password protected key without bothering you too much in practise.
If you need to pull on a remote that doesn't have your private keys (as is good and proper) you can run ssh -A foo@bar.com to take that identity with you onto that remote (e.g. so you are able to pus/pull from there).