The part that makes me uneasy is when the AI writes both the code and the tests. They might agree with each other and still miss the same thing. How are people here checking for that without doing the whole job twice?
I've started using Stryker. We have a CI job that mutates the code before running the tests. In the first run, it has returned hundreds of code changes confirming that the tests are biased. I haven't yet fixed those issues, but for now i am banking on this approach.
Occasionally, Claude may not be able to fix the issue – and it may turn out that Codex can, because it has a different approach.
Either way, at least two LLMs is the only way to get (mostly) reliable code.