You should encrypt your private key with a password. Then you can use ssh-agent to hold a copy of that decrypted key in-memory so that you don't have to type your password in every time you 'git push'
But there are also niche situations where an attacker has read permissions for files but not memory. It's atypical, but in cases where you have 'confused deputies' it happens - for example, if I have a service that attempts to safely broker reads to files but that service has a logic bug where a client can trick it into reading sensitive files.
The classic example of this is a path traversal in a web service.
Also, while there's no boundary, attackers are a lot happier reading files than memory. They're only human.
In general I find that, when you can, you should just encrypt your data. It can protect you in some surprising ways.