Evals will break

The argument in the article is backwards. Evals test the stability and boundaries of a concept. They are not created before the concept has been prototyped (which the author acknowledges).

An eval is not somehow breaking silently due to some new capabilities in an LLM. It wouldn't be a good eval if it did. What it does is steer the LLM towards specific goals. If anything, an argument can be made that they restrict creativity and experimentation by narrowing goals.

If the argument is that evals need to written before some new behavior can be devised, that's incorrect. There are an infinite number of evals that test for things which cannot be done. Only when something has been demonstrated to work in a specific context, can an eval be written.

I read through this blog post and it's timely given how close the models are to max out the benchmarks/evals.

One thing which was not addressed but will be interesting to discuss would be benchmarks/evals that conflict.

Are there desirable emergent behavior that might not be optimized because the evals penalize them?

AI slop

“Eval” is not testing. This post is written as is no one ever heard of a thing called testing.

For the uninitiated: if you think testing is nothing more than simple operations and assertions, then you don’t know anything important about testing.