This sounds like a wrong design decision. I wish nobody could log into my github account using anything but my SSH keys.
This is also true of my AWS account: my ec2 instances are protected by SSH keypairs, but if anyone gets my AWS password, he has full control over everything.
I'm not a security expert, but SSH keys feel way safer than passwords, especially with all those recents article showing how easy it can be to bruteforce passwords.