Can anyone think of any lessons to draw from this for normal development, where we don't have oracles to serve as guardrails? I write specs but the agents still find ways to insert bugs between the lines. Oh well, job security.
Normal development isn't so lucky, you do care about code quality. So you have to review the code the agents write and handle the testing yourself. The agents will not do what you want, they'll do what they think you want, and you're the only person who knows what you think.
Obviously matching functions byte-byte is not it, but I think with enough creativity, you can have guardrails and oracles almost everywhere.
E.g. before diving into feature development, you can have agents generate a test suite based on your implementation first. Then, when developing features, this test suite shows what broke and agents can double check if that was intended.
How that test suite looks like obviously highly depends on the environment.