The Bitwarden desktop app has a built in SSH agent [1], which lets you store SSH keys inside Bitwarden. I have set the app to always ask me for confirmation when the key is requested.
If you're using say yubikey for hardware ssh key (via gpg ssh agent/smartcard interface) you can also make it require physical touch of the key to confirm every operation
first time you see the light blinking without doing anything intentional is a bit of a sinking feeling
It's mostly the default ssh-agent that has a different security model.