that's interesting. what example problems incremental refinement isn't able to solve?
As far as a specific example, I remember a discussion with Uncle Bob where he specifically mentions banking and accounting as systems that shouldn't use that approach because you'll build the wrong thing.
But you're right, evolving code often turns it into a mess. The only way that doesn't is if, at each stage, the people working on it keep the architecture and code clean. That takes discipline, not just by the programmers, but also by management - they have to give the programmers time to do the cleanup that is needed, not just time to shoehorn something in.
If you build a 1-story house, then gradually do what-it-takes to build another floor on top of it, you'll either never have a tall building, or end up with something like a pyramid or huge angular support pylons (i.e. extremely wide base).
You may end up getting stuck on a local maximum in the software design space.
To optimize and really find the global maximum often requires backtracking, which in the case of software development might mean throwing out large portions of the codebase and starting over.