I actually started a collection of annoying bugs I’ve seen in the wild. I give the llm the buggy implementation and ask it to write a test that catches it. So far not even a frontier model (Claude Sonnet) can do it, even though they can find and fix the bug itself.