With AI, the new evolution in software is to skip the "telling what to do" part and directly specifying the goal - "telling what to get done" and let the system figure it out.
Not exactly. Writing software has always been "telling computers what to do". Using AI to do this doesn't change much. In the very early days (after CPUs were invented), you told the computer what you wanted to do with CPU instructions directly, so you had to figure out how to manage memory and all other tedious things needed to get your program to work. Later, higher-level languages were developed to abstract the complexity of the hardware away, so you could tell the computer what you wanted in higher-level terms without so many low-level details. An AI tool just takes this to another level.
The possibility of pure goal-oriented systems are at an inception now. So it's a totally different game, different mechanisms, and so on.
Thinking the user of AI can just spec a goal and then the AI automagically produces perfect results is just the pipe dream of the lazy who won't ever have a clue about how to verify the resulting system, much less the system that produced it. Good luck with that!
Software is about 100% correctness, which is damn-near impossible. And when it fails, it will fail at the worst time and in the worst way possible. It's just a matter of when, not if. We can't even remotely do that (for nontrivial systems), but we're gonna make a machine that can do it?!
Shhheeeeeiiiitttttt.
- For the past 50+ years, really intelligent people, including Turing award winners have believed in this dreams - In fact, getting computer to solve problem on their own is the more intense path IMHO. The lazy path is sitting content with the way things are.