Interestingly it appears this Authy method does not use PAM. I wonder if there is a reason for that, since a PAM module would be a good deal more flexible (this sort of thing is what PAM is for after all).
Also, the way I would intend on using this is such that if your public key is in authorized_keys, you log in like normal. If not, you authenticate with PAM using both your unix password (or ldap or whatever you have configured) and your 2nd authentication system. I log into one or two of my servers from strange computers often enough that disabling password authentication entirely on those servers is limiting.