This sounds cool, but it's needlessly complicated. Why not simply configure SSH to use multiple forms of authentication [1]: password + public key auth + 2FA (Duo Security, Google Authenticator, Authy, etc.). That's all you need to achieve a very secure state.

1 https://wiki.mozilla.org/Security/Guidelines/OpenSSH#Multi-F...