Nevertheless, doctors in many areas are in short supply and it's incredibly hard to become a doctor. Why? Because for the 5% of hard cases when you absolutely need an expert or else there are serious consequences, and it's very hard in advance to predict if someone's case is the 95% of trivial grudge work, or that 5% of critical work.
Software development is the same, the vast majority of the work is trivial grudge work and there's a lot of that grudge work, enough of it to keep millions of people employed. But there's still that 5% of the time when you need to be an expert, when you need to know data structures and algorithms and good architecture and have solid knowledge of your domain. The problem with software development is that most people just take that 95% for granted, get complacent and think that the 95% of trivial work is the job and that they don't need to be prepared for the remaining 5%.
My opinion, and that's all it is, is that in the future there won't be a need for that 95% of work, it will be automated and all that's left will be the remaining 5%. The people who became complacent and came to believe that their job is the 95% of trivial work will find themselves in a very difficult position.