This is an emerging space with lots of interesting tools coming out... There are many established benchmarks out there (i.e. included on front page of llama2 release), but most product builders have their own sets of evals that are more relevant to them...

Here is a thread exploring differences between llama-v2 vs. gpt3.5: https://twitter.com/rajko_rad/status/1681344850510376960