Seems like a non-sequitur? "because math is formalizable", I agree it makes sense to then predict that data-driven AI models will be able to perform well on it. But where is the evidence that software coherence is formalizable? If it were then I would agree, but to me it seems that coherence in a software system relies on a ton of intuitive reasoning and familiarity with company culture, past failures, etc. it's like the least formalizable thing there is.
AI improves over time, sure, but it has differences and advantages and disadvantages over human intelligence. It's just a fundamentally different thing. In the same way calculators are superhuman, and improving, but can't write books. You yourself acknowledge to some degree the uneven nature of that improvement when you referenced formalizability.