This question is slightly different because you have to track a testing set and the metrics. I would look to the many ML workflow tools
To rephrase, I am looking for a tool to do model lifecycle management https://github.com/kelvins/awesome-mlops#model-lifecycle and wonder if there is any one in particular that you'd think is better suited for prompts, i.e. an array of objects with templated text