Most tasks receive the interpreter binary which the coding agent can run to test their code (though the binary itself is adversarial too, there is a fun easter egg in there if anyone wants to give it a try). Some tasks just get the spec and a piece of code to understand. Some models interpret the code/spec incorrectly so fail that one.