I've found that context is everything to getting consistently good output, and augmenting your prompts with known truths with SERPAPi, embeddings and a vector db, brought really flakey results into the >90% accuracy threshold.

As an aside - does anyone have good tools or methods for testing and evaluating prompt quality over time? Like performance monitoring in the web space, but for prompt quality. The techniques to use LLMs as evaluation tools of themselves always has seemed flakey when I've tried it, I'd like to use a more grounded baseline.

For example, if you have a prompt that says "What is the weather today in {city}?", you can run it against a list of cities and expected outputs (using a lookup to some known truthful API). That way, when you make changes to the prompt, you can compare performance to a baseline.