Ask HN: How do you version GPT prompts

1 pointokoma5 comments

This is for the purpose of tracking improvements of a code completion or a chatGPT prompt.

Are you using an existing solution, rolling your own or a mix?

I found a bunch of resources here: https://alphasec.io/prompt-engineering-playgrounds-marketpla...

Good candidates for what I am looking are OpenAI Playground (once it supports versioning and variables for chat), PromptLayer (once it supports chat prompts), EveryPrompt (chat prompts missing). More dev oriented, LangChain https://js.langchain.com/docs/getting-started/guide-chat which can then be easily versioned.

Git

What structure do you use if you wanted to check the accuracy of an older prompt with an updated or extended data set?

This question is slightly different because you have to track a testing set and the metrics. I would look to the many ML workflow tools

Thanks for pointing towards the right direction. I'll edit the original question.

To rephrase, I am looking for a tool to do model lifecycle management https://github.com/kelvins/awesome-mlops#model-lifecycle and wonder if there is any one in particular that you'd think is better suited for prompts, i.e. an array of objects with templated text

Reply on news.ycombinator.com