Protecting SSH Keys with macOS Secure Enclave
gatezh.com
gatezh.com
When I installed WireGuard on macOS, I was happy to discover that it stores private keys in the system keychain. Accessing the private key requires Touch ID.
Then one day after a system upgrade, the WireGuard app crashed on startup with:
[APP] Unable to open config from keychain: -25300
Then the app deleted all profiles because they are all "orphaned", apparently for "keychain consistency" [0].From then on, I just use a text editor to edit my WireGuard configuration files, and import them to the app. The bug still exists to this day [1].
[0] https://github.com/WireGuard/wireguard-apple/commit/dfb685f2...
[1] https://www.reddit.com/r/WireGuard/comments/wyt828/osx_wireg...
A lot of the Security Framework came from Classic, hell, the idea of a keychain came from Classic! Just a mess.
You linked a commit and a Reddit post. Is there a proper Github issue as well?
Edit (March 2025):
Wireguard, to my surprise, has no bug tracking system like Jira or issue tracking area such as GitHub.
If you want to see this fixed, the only way to instigate change is to write team@wireguard.com and jason@zx2c4.com.It's funny you mentioned that. The GitHub repository is a mirror of https://git.zx2c4.com/wireguard-apple. You cannot create issues there. You're supposed to send a bug report to the mailing list, which is enough friction that none of us did so far.
Emailed per https://news.ycombinator.com/item?id=50450200
Definitely still a bug right? Linked them to your parent comment.
Edit: oops sorry team, think you're on it (4d ago): https://github.com/whyredfire/wireguard-apple/commit/0bac142...
Guess it can serve as feedback on the friction when issues can't simply be opened though!
Thanks! Yes, it is still a bug.
> Edit: oops sorry team, think you're on it (4d ago)
I don't think @zx2c4 has seen this PR, and the author of the PR has not sent a bug report yet, so it's probably fine :P
Its unfortunate no updates for Wireguard macOS have been pushed out for 3 years, App Store is still showing 1.0.16 as the latest version – as also confirmed on the Wireguard website at the latest version.
No doubt macOS APIs have moved on a lot in 3 years and so perhaps bugs that are being seen are simply a reflection of the lack of updates to keep up with API changes ? Just thinking aloud here, I don't know if the concept of OS API changes are applicable to this particular PR / bug.
I appreciate when both options exist, because I've passed on reporting so many bugs, because I refuse to create a Microsoft account to do it.
In fact, I don't understand how sending an email is high friction.
99% of the time it's perfectly fine to just email the maintainer with a bug report and maybe a patch.
> In fact, I don't understand how sending an email is high friction.
Judging from the fact that after so many years, such a widespread (as seen in the Reddit post and here) issue still hasn't resulted in a single bug report in the mailing list, I'd say most people (including myself) do indeed consider using a mailing list to be higher friction.
If I'm not mistaken, unless you're already CC'ed by a participant, you'd have to subscribe to the whole mailing list just to receive notifications for a single thread. For an "issue" on a Git forge, participation is usually as easy as upvoting and then clicking the "Subscribe" button.
In the current age, its useful because AI will submit tickets using the CLI that don't use the form, so I can immediately close and lock them and go on living my life.
And features built for normal people
The overlap is very rare
For platforms where development is unhindered by arbitrary obstacles like these and others, wireguard works fine. :)
All of the traditional components of VPN management are removed from the core code. Wireguard follows the model "do one thing, and do it well." It's also built into the kernel, which inherently can make it faster than userland VPN software.
This is why companies like Tailscale exist--as they're basically selling back the features that you would get in a more traditional IPSEC VPN, but in front of Wireguard instead.
SSH keys in plaintext was one of the first thing that bagel [0] our developer endpoint secret detection focused on. Then we embedded bagel as a library in our developer endpoint protection [1].
[0] https://boostsecurityio.github.io/bagel/ [1] https://boostsecurityio.github.io/dazio/docs/install/
This is why I'm skeptical that the approach can scale reliably, even though I'm relying on it today. (Yes, PATs would fix this, but the enclave model is what I'm trying to retain for now.) What I've done instead is wrote a small daemon that sets the SSH authentication socket and does agent forwarding. This works when I'm traveling with laptop...but not yet from my mobile phone.
Example:
Laptop Remote Mac
┌─────────────────────┐ ┌──────────────────────────┐
│ Secretive │ │ ssh / git commit -S │
│ (Secure Enclave) │ │ │ │
│ ▲ │ ForwardAgent │ ▼ │
│ └── ssh ─────┼─────────────────►│ ~/.ssh/agent.sock │
│ │ │ (stable proxy) │
└─────────────────────┘ │ │ │ │
│ ▼ ▼ │
│ forwarded local │
│ laptop Secretive │
│ agent (if you sit │
│ at this Mac)│
└──────────────────────────┘
NB: I'm not using Rex instead of herdr and because Rex uses its own protocol, I always need to have one SSH tunnel to reset the remote agent socket. Not optimal, but it works. All of this feels wrong from a UX perspective, and it feels like I'm holding on too hard on secure enclave for what my use cases are. Feedback appreciated!I’m a happy user!
1. The Secure Enclave being inaccessible when you're not present is a feature, not a bug. If you need to run headless then its not the correct solution to your problem and you'll have to find another.
2. You don't need an SSH agent proxy, you can use a `match` block in your `ssh_config` with something like `host * exec "test -z $SSH_TTY` to check whether you're operating over an SSH connection and select Secretive's SSH agent if you're using the machine locally.
I've started building a timeout-based ssh-agent population utility, but if I'm honest, TPM keys without user presence is probably sufficient.
I'm not using the secure enclave, but I've been using YubiKeys for many years, and now that AI has arrived, I consider the above to be a good thing.
This means, for example, that unless I am physically present with my key inserted, activated, and touched on every operation, an AI agent cannot access any of my production servers. As I do not want AI agents to access my production servers, this is a very good protection technique. I am assuming that no matter how well you try to sandbox and restrict your AI agents running on your local machine, they might eventually be able to access any file in the file system.
I found I was getting fingerprint fatigue. Do I need a fingerprint request every time my dev server wants to connect to dev Supabase? No. That's expected, and zero risk. There's no data there.
I worried that a nefarious request would be auto-approved by me. When you see 20 requests a day, your muscle memory just reaches for the sensor.
So I have a separate 1Password vault that a service account [0] can read. That token is in my environment as OP_SERVICE_ACCOUNT_TOKEN, which the `op` tool understands.
It also has access to a vault with 'AI-safe' keys that I want Claude to be able to access without asking me.
Now, these requests work invisibly. And when I see a fingerprint popup, it's unusual, and I don't auto-approve by habit.
I also keep an item called '_One-day keys' in a separate vault. (The underscore just pushing it to the top of the list for easy access.)
It has prod keys, mostly restricted read-only. If required, I move that over to the open vault. Claude knows this item and has instructions to remind me to move it back out when it's finished what it was doing.
MCP for this would feel like I was giving up control? I don't know exactly what that thing is doing. Whereas I know exactly how my configuration works.
YubiKeys were the only solution that fit these requirements, so I went with those. I have been using them for the last ten years or so and I've been very happy with this solution. See https://github.com/drduh/YubiKey-Guide for a description of how to make your RSA keys stored on YubiKeys work with GPG and SSH at the same time.
Relying on a small set of ssh keys for a longer time is anyway not that great, it's best to have some mechanism to cycle them.
I wouldn't use any automated sync tool for that, just a script that ssh's into all the servers and replaces/edits authorized_keys if something changes.
I would never use something like this (https://github.com/shoenig/ssh-key-sync). It basically gives Github or someone who compromised Github access to all your systems.
The author says he doesn’t like the idea of having keys sitting on his file system ready to be snatched by bad actors.
That’s why they should be passphrase protected.
You use ssh agent to store the keys in memory and for convenience you can store the passphrases in keychain. MacOS OpenSSH has the UseKeychain config directive for that.
I use Yubikey/FIDO2 myself, but not because private keys cannot be protected on a file system.
Assuming the attacker has user rights (which it needs to read the key) it could also read the passphrase, right?
Update: I tried it out and stand corrected. ssh-add doesn’t store it in the user-readable keychain. So the passphrase is protected. (And also not recoverable by the user).
It's effectively an open source version of a YubiKey, the hardware itself is all open source and it handles ed255519 SSH keys really well.
inb4 they're not the same, I know, I really like what the TKey team is doing.
But I think I would take this over a yubikey. I really disliked the fact that one usb port was constantly in use.
The idea is precisely that you keep them permanently connected in the laptop.
SSH private keys should be passphrase protected.
You don’t have to do that if you use FIDO2 with a Yubikey.
It’s a mind shift worth considering.
You then can accept that your ssh keys are different per hardware token (because the private key cannot be extracted).
No more sharing of private keys in your password manager.
I’ve grown to appreciate ssh should only be used with git for signing commits and as a deploy key (eg in a CI system).
I always use https for day to day git operations. My keys reside in gopass and are extracted by gopass-git-credential when needed by git.
I use that within devcontainers
Better use short lived tokens with auto refresh. For example OAuth tokens. They can be used for the same use cases like PAT, but they expire after a max of 2 hours.
It would be nice if GitHub or GitLab could limit a token to a given IP, but with how proxies and dynamic ips work I would imagine it would be a hassle to get it working correctly for everyone, but it’s better than nothing.
> https://news.ycombinator.com/item?id=46025721
One of the problems I constantly face is that the SSH configuration is not portable when I use these esoteric things. Even with Secretive (which plugs a simple SSH socket), remote agent forwarding sometimes fail. (ie. Due to the old version on the server or configuration issue, or something else setting SSH_AUTH_SOCK...)
Moreover, it is not possible to drop these files into a file-sharing folder, like OneDrive or iCloudDrive or SMB share, and share them with many hosts at the same time. (Especially when different operating systems are involved...)
I found the hacker way simplest and easiest; put a basic Makefile with bunch of rules, have a "central" host/machine to distribute the keys and configuration from that point on...
On iOS/ipad/(also macOS) you can also use the SSH terminal emulator Termius which has support for SE keys built in as well. So I have the set of keys synced to my VPS, this way I know that only my physical devices can log in and nothing else.
Host github.com
AddKeysToAgent yes
UseKeychain yes
IdentityFile ~/.ssh/id_ed25519
Docs: https://docs.github.com/en/authentication/connecting-to-gith...To answer the question a few people asked: I ended up using Secretive. The native approach works, but I couldn't live with the Touch ID prompt it shows. Secretive keeps the keys in the Secure Enclave too, with a much better prompt and per-key settings. I'm writing a part 2 about it, including forwarding the agent into a dev container so the key never enters the container.
It's often the case that we make much effort into securing the key but expose it via an open agent socket at a well known path or use an open control master path for multiplexing.
Certain automation and dev tools like Ansible make it a pain to work if agent key requires a confirmation.
Claude and Cursor love to do `git ls-remote` at random times, which makes 1Password show a prompt asking me for permission, even though they initiated it from a non-interactive shell.
So now they just fail silently at the password prompt over and over again.
It uses either the TPM with Windows Hello, or an external security key.
It might require downloading a newer Microsoft openssh release to generate the keys, but the current Windows 11 ssh can use those keys.
https://github.com/unreality/nCryptAgent works quite like Secretive
https://github.com/PowerShell/Win32-OpenSSH/issues/2040 https://github.com/PowerShell/Win32-OpenSSH/issues/2408 https://github.com/PowerShell/Win32-OpenSSH/issues/2413 so there are still some quirks at the moment
Download the newest release here: https://github.com/PowerShell/Win32-OpenSSH/releases. Just use ssh-keygen from the release to generate the key, the key itself works fine with the older ssh shipped with Windows.
Some hardware features must be present for it to work. But I guess all (or most) Windows 11 compatible devices will do fine.
This means the machines you want have access to SSH (i.e. GitHub push/pull), but none of them have an actual SSH key - if a machine is compromised you drop it from Tailscale. More of a security/usability compromise than what OP is looking for, but solved a lot of frustrating paper cuts for me - I use it every day and I never think about SSH agents any more, it all just works
Recoverable via console access but disruptive.
Still using Tailscale but have redesigned my setup and keep coding agents on a short leash. Also switched to internal Forgejo for speed and use GitHub as a push mirror.
I like a lot of the ideas in Tailscale, but I think I adopted too much of it in a fug of excitement and power.
It is not one of the cryptographically safe curves, likely intentionally so.
Saying it is cryptographically unsafe is a wild exaggeration of what he says.
[1]: "Claude Solves the NIST Curve Seed Mystery: It Was Timestamps" - https://cellcog.ai/blog/claude-ecdsa-curve-seeds/
My original thought was that people often confuse complexity as security. Its added friction layer for sure, however it cuts both ways, makes slightly harder for every day hacker, the monitoring becomes harder as well.
Work on macOS 26 and later.
This is the most recent one I found via Googling, I have not actually used this product:
I think this is incorrect and the "ssh:" at the end is indeed a comment.
Definitely a recco from me as well. -- git hub here: https://github.com/maxgoedjen/secretive
> Any malicious script that gets access to my filesystem could just copy them and get into all my servers.
So this is postulating a "malicious script" that has access to your filesystem but not the running system. Because if it's running on the actual system it doesn't need the key, it can just execute your remote stuff on its own. Or it can get the key by MitMing you by making your next "ssh" invocation steal your passphrase, or just read the unencrypted key out of the ssh process, or even just reimplement ssh to do the SE negotiation on its own. Or...
Like, there's near zero added security here. The only scenario I can think of that would have any meaning is someone getting access to an offline image of your filesystem, an attack already prevented by drive encryption (and, perhaps, already defeated by insecure backups!).
You see occasional attempts to use TPMs for similar purposes, and they're similarly silly. People need to understand that the TPM/SE exists for the benefit of third party authentication. It's not for you, the operator. It's there so that your bank/employer/intelligence-agency/whatever can know that the human being doing authentication is using the same device they've been known to use before.
Presumably biometrics are required to use the key, so they have to convince you to prove presence when you aren’t expecting it?
> Or it can get the key by MitMing you by making your next "ssh" invocation steal your passphrase
What passphrase?
> or just read the unencrypted key out of the ssh process
The key never leaves the SE, so this is not possible.
> or even just reimplement ssh to do the SE negotiation on its own
See first point and biometrics.
No more passwords to enter anywhere, and whatever an (AI) agent or malware is trying to do will require a human interaction at least, and not work when the screen is locked. The public keys authorized for this are distributed to the fleet via puppet, so we get control over who gets to sudo permissions and can easily provision or decommission machines.
Pretty stoked about it so far.
There’s no reason the private key needs to be stored in Secure Enclave. Your passkeys are in the MacOS Keychain, encrypted using the Secure Enclave. They are decrypted when iCloud transfers them to a new machine.
A similar protocol could be used with ssh keys. Keep the keys on disk as usual, making them easy to back up. Encrypt them using the enclave. You can use the Secure Enclave to decrypt the key for backing up (with a backup-specific secret).
This way all of the protocols and tooling remains the same, except for obtaining the SSH key encryption secret from Secure Enclave. Backup, ssh login, ssh agent all use the same keys.
Do what Apple does. Making the process more difficult actually makes security worse.
Unless you specifically mean (f)TPMs/secure envlaves.