You're almost there, keep going a little farther, and you'll have eliminated passwords and invented FIDO.
Do you have any idea why this is not popular? Is it too hard to implement or is it just that business's do not see security as something to invest a lot in?
https://www.yubico.com/solutions/fido-u2f/
https://webauthn.guide/ for an intro
https://www.w3.org/TR/webauthn/ for the JS API