The problem is that we generally rely on human intelligence to fill in gaps in specs that are really not stupid for a human. AI will exploit gaps that a human would basically say aren't there until they see evidence to the contrary.

So we can say that's a bad spec if we like, but what's the answer that leads to? I don't need an AI if I can just declare all users must be really good programmers spending their days writing unambiguous specs. That wasn't really the goal of the AI though.

Unfortunately, when gaming metrics has real world consequences (and this is of course the whole point of those metrics, assuming they're not gamed), then things like 2008 happens, and tens of thousands of people lose their homes.