One can set a password on the ssh key, but I still felt a bit paranoid about it. I found a way out with fine-grained personal access tokens which allow you to choose the repositories this token will have access to [1].
My setup consists of two ingredients:
1. GPG encrypted fine-grained PAT: `gpg -c --no-symkey-cache --pinentry-mode loopback my_name` ends up into `my_name.gpg` secret.
2. A git credential configuration which is generic across git repositories:
[credential "https://oauth2@github.com"]
helper = "!f() { test \"$1\" = get && echo \"password=$(gpg -d --pinentry-mode loopback --no-symkey-cache $_GITHUB_TOKEN)\"; }; f"
Now switching identities results into setting the env var `$_GITHUB_TOKEN` to the path to my gpg encrypted token, which will be decrypted by git on the fly. You can figure out a suitable way to alias this for yourself :)And it only activates for git urls of the from "oauth2@github.com" which allows you to clone public repos without questions.
Another advantage is that you can share the same repo with other people, no need to maintain a copy.
Disadvantage is that you have to enter password each time you push/pull.
[1] https://github.blog/2022-10-18-introducing-fine-grained-pers...