If you use fail2ban make sure to pin the right key to the right host. Otherwise ssh will try all the keys and get you banned from your own host. The easiest way is to use the ~/.ssh/config:
Host myhost
IdentityFile ~/.ssh/myhost