If someone has access to a shared secret and then shouldn't you should assume the secret is now compromised and rotate it. Rotating the keys doesn't solve this problem.
It's a bad idea to store secrets in any form in your source code repository.
Obviously this is a huge hassle and isn't easily done with all kinds of secrets (which is what I think you're getting at?). But it's also often necessary.
This doesn't have to be a Git repository, nor does it have to be the same repository as the code which uses the secrets.
> it's incredibly helpful to store the configuration along with the code
It's helpful, but ultimately means that the ciphertext is potentially available to the world and existing keys may decipher it in perpetuity. Thus this is not a recommended tradeoff to make.