Integrate it with your CI/CD pipeline. I have a Gitlab pipeline that runs kustomize and pipes the output to kubectl for every deploy to do things like setting secrets, image tags, creating ConfigMaps from config files in the repository, etc.
That's essentially what we already do. My original point was based on the fact that the config changes all the time (as opposed to 4 times a year).
What are you changing all the time in your config?