They can still get you if you invoke SSH on the remote, as the password is sent to the remote machine one character at a time, then forwarded on to your ultimate destination all at once.
This will explain it better than I ever could: http://www.unixwiz.net/techtips/ssh-agent-forwarding.html
They could potentially use this information to know the length of a password, which would make brute-forcing easier. A very hypothetical attack, but fun to think about! Less effective than a $5 wrench, no doubt.