Here's one way to screw it up: make the public key the password. Normally only the user and the site know the user's public key for a given site, so it's as secure as a password, and nobody notices the problem.
I'm sure that there are big enough fools out there, I'm just thinking there might be signs they are doing something wrong in that case.