In fairness, I do too, but I keep a dated set in a subfolder (`archived` if it matters) and when I edit I start with `current.doc` (unfortunately Word is still the preferred format outside of... anyone with any sense), save it as a dated version.
This way I only ever care about `current.doc` (and its exports into real formats) but can still reference stuff later.
That said, I've also experimented with simple resume data formats (primarily JSON and annotated Markdown) that got transformed into doc and real formats, and kept that in a normal repo. It was better, but since it was a homegrown format, annoying in the long run. I'll be revisiting it.
The nice thing about transforming a custom resume is that I was able to (more or less) automatically re-target it depending on the type of position I was submitting it for. That was only semi-automated, but spoke of some promising ideas.