Jumping in based on our experience in case it's helpful:

- Evals are very useful and a core part of the best practice for creating LLM apps

- There are already excellent solutions for model and prompt eval (etc.) including Parea from YC and many others