The benchmark I care about the most for my development workflow is on structured output.
Paul Gauthier made this benchmark [1] to measure correct git diffs. If you ask GPT-4 for help with your code, it can output a change in a git diff more reliably than 3.5.
My hope is that we can do that with Llama 2.