The device could instead have a display that displays a password, QR code, one-time-key, etc... and then you would scan it using a webcam or your phone (or type it manually).
Yeah, but that way it's displayed out in the open and you can make a mistake copying it. I think just exchanging usb flash storage sticks is preferable unless you sufficiently distrust the system you're on.
Maybe like a phone running a TOTP application?
I would have more trust in a separate device that has no networking capabilities.
What if it's in a "secure element" whose hardware and firmware make it provably incapable of divulging the private key?
I don't think that's possible... it reminds me of Apple saying that they can't decrypt your phone, but it is decrypted when you use it and many apps have permissions to transfer basically everything over the network (and Apple has root too).