This tangentially reminds me of a very interesting idea I think I encountered in a Borges story - for a map to be useful, it must be inaccurate.

That is to say, a perfectly accurate map would just be a replica of the the terrain it is mapping - anything less would technically be inaccurate. Only by leaving things out and "summarizing" an area does a map become a useful representation of the terrain.

It really depends on the program. Some problems derive most of their complexity from trying to correctly model complex real world rules, like calculating pension payments for employees that were hired at different times, for different positions, under different union agreements, having worked varying numbers of years.

Some problems have short, clear definitions yet remain challenging. An algorithm for solving them may have been discovered decades (or even centuries) ago. But trying to produce more efficient solutions can be a life's work and more. One of the most obvious examples is integer factorization, which had a basic algorithm published 800 years ago yet is still a research problem today:

https://en.wikipedia.org/wiki/Trial_division

https://en.wikipedia.org/wiki/Integer_factorization#Factorin...

The was (still is?) one of the major issues with outsourcing development work. External contractors tend to need excruciatingly detailed requirements to produce the expected results, because they typically lack the domain knowledge to guess correctly when faced with ambiguity.

Very often this means getting into long cycles of iteration. If time zones are distant, those cycles can be very long and painful.

Perhaps outsourcing to AI will work much better, as those iterative cycles can be done in near real-time. Maybe there needs to be many different models, each with some industry/domain specific training to increase the odds it’ll make the right guess when it needs to.

With almost all software you're likely to write professionally, yes. There are edge cases (as there are with all things), but those are not things AI is good at to begin with :)