That depends on how you authenticate. If you pass your SSH password on the command line then anyone on your machine doing a `ps` at the right time could see your password.

I find using a ssh-agent to load password protected SSH keys works best.