This leaves out a possibility of a MITM. An attacker can steal the unencrypted machine host keys and pretend to be your computer. And since you're entering a clear-text password, it's easy to sniff.
Moving the host keys into hardware root-of-trust would help. But macOS Secure Enclave barely supports that, and it's also pretty slow.
But then I also realized that it's still likely to be hard to access for the attacker. So I don't really have much issues with that.
Really? Secure Enclave supports only one asymmetric algorithm. With only some limited usages.
What you can do though is use Secure Enclave powered app for storing and managing access to the keys. So basically app like "secretive" run on your normal OS, but isolated and only it can access keys, use them and there no export function even with admin privileges.
AFAIK this will fail if there is a local root exploit on macOS, but still much better than keeping keys in plain text.
But that's it.
Anything more complicated is not possible. You can't even upload your existing key into the SE.
So "secretive" and similar software is not as secure as let's say hardware token.
If I'm wrong please correct me, but when I researched the topic I come to this conclusion.
This is the initial inspiration for Secretive: https://github.com/sekey/sekey - it uses the SE to generate and store the actual private key, so it never leaves the machine. Hence its limitations.
2. The notion of someone having access to / compromising your device in order to capture SSH creds doesn't strike me as realistic