Careful when SSHing into an untrusted host. They can find your public key(s) that way and maybe identify who you are on GitHub.
My first thought is to create an entry in my ~/.ssh/config with an entry for the untrusted host I'm going to ssh in to, using the "IdentityFile" keyword to point to some bogus/misleading identity file. How does that sound?
Host *
# Do not send pubkey to each and any host
PubkeyAuthentication no
# Use Agent
IdentitiesOnly yes
Then override 'PubkeyAuthentication' to yes for each known server you want to connect to."Privacy: a secret matter."
"Privacy: freedom from unauthorized intrusion."
"Privacy: the quality or state of being apart from company or observation."
So you see, privacy IS security.
Not just any public key, it's your public key. It identifies you.