trywale.com
Today we're launching a tool to help you evaluate and test prompts for Generative AI.
We have been building different GPT3 apps and noticed a gap in tooling to help developers assess the quality of different prompts.
Our tool helps you template and test on different datasets and LLM models like GPT-3, GPT-3.5 and open source models like flan-t5-xxl.
We are just getting started and would be delighted to receive your feedback.
I do some prompt engineering on a daily basis, and I find that the challenge is often less about the writing and more about ensuring that the output is what I want, and consistently so.
Trying to think of some useful features for this tool, I'm thinking that some improved tooling for testing output could be useful. One thing is to validate the output, but also to be able to run the same query in bulk to check that the output always validates correctly. The model often outputs something correctly one time, and incorrectly another time, for the same prompt.
Just some thoughts from my side!
One of Wale IDE's features is to run your prompts over an CSV of data - see "Import data or start from scratch" on our landing page. Give this a try and see if it helps with testing your prompts.
Let us know if you would be open to a short 15 min call: link: https://cal.com/zach-zhao/20min?duration=20.