I honestly don’t know where you got that idea from. I always thought the whole point of computer programming was to solve problems. If it makes things more complex as a result, then so be it. Just as long as it creates fewer, less severe problems than it solves.
And this is where the increased complexity is necessary for a solution, not just Perl Anti-golf or FactoryFactory Java jokes.
I don't think you can get away from this - yes, can solve a problem, but if you model problems as entropy, increasing complexity increases entropy.
It's like the messy room problem - you can clean your room (arguably high entropy state), but unless you are exceedingly careful doing so increases entropy. You merely move whatever mess to the garbage bin, expend extra heat, increase your consumption in your diet, possibly break your ankle, stress your muscles...
Arguably cleaning your room is important, but decreasing entropy? That's not a problem that's solvable, not in this universe.
In an isolated system, entropy can only increase. Moving at all heats up the air. Even if you are exceedingly careful, you increase entropy when doing any useful work.