I wish SSH had a narrow kind of delegation capability... E.g. if instead of ssh-agent having access to the key, it used the key to sign a tuple like {time-limit, [host-ip], ephemeral_pubkey}, then forgot the private key, and used the ephemeral key to actually authenticate to hosts— but they'd only accept it from the listed host ip set and during the specified time limit.

Obviously if the attacker has access long enough they'll capture the passphrase and private key eventually, but it would be nice to narrow the window.

Right now the best you can probably do is use a gpg smartcard with the ssh-agent/gpg-agent shim stuff, (random tutorial here: https://github.com/herlo/ssh-gpg-smartcard-config/blob/maste...)