The real problem is how do you ensure that code alterations preserve observable behavior. Complex systems generally have complex specifications.
Possibly, but if Codex is like similar AI text generation tools, complex systems or large software architectures with many interacting components are going to specifically be the corner cases where results are less reliable.