It is better to have simple solution at start, then, after each breaking feature, whole thing needs to be refactored, because initial assumptions about that software might have been changed.
I think the key is that you accept that this is true, and that having a good architecture is a continuous process that never ends. You build the best architecture you can imagine, given the current level of knowledge. Then you have to be willing to refactor every single day after that, as you gain new knowledge. Too many people think that refactoring is only for special occasions, when things have gotten really bad. Every single PR can contain a small refactor. Small changes can accumulate and eventually lead to major architectural shifts.
Quick throwaway code is orders of magnitude better (unless you're landing airplanes or building x-ray devices). Especially if you're consciously treating it as a throwaway.
The only way to build well structured system that lasts long is to have vast domain expertise, meaning having done literally the same thing multiple times in the past. This rarely happens in general, and pretty much never happens if you're innovating.
When something is "the same as some other piece of code with different variable names" it becomes it's own isolated solution, be it a library, framework, service, or software. We typically don't run our own SMTP servers, don't write our own databases and don't implement TCP/IP stack from scratch.
When we're not doing all those things, typically what's left is innovation: i.e. building something that doesn't have an existing generic solution.
What if you had the ability to build a system that was initially structured well enough such that it could be made to last indefinitely? From a cost/benefit standpoint, is the Ship of Theseus approach not the most ideal for a business owner?
Even for a developer, the notion that you have to constantly "trojan horse" your new product iterations into production via the existing paths means you will achieve mastery over clever workarounds and other temporary shims. Once you gain competence in this area, it is possible that you will never want for a new boat again. You start to get attached to the old one and all of the things it takes care of that you absolutely forgot about years ago.
If you could have that, it would obviously be amazing. Do you (or anyone) have that ability though? So far it seems the answer is no. People are just not smart enough to predict the future.
As a more nuanced answer, a system that is scalable enough to grow to 1000x the current load is usually way too expensive to build with the resources you have "right now". The best you can do is build a system for 10-100x the current load and hope you haven't forgotten any edge cases, but usually you encounter them way sooner than 10x the load. Building so that you can easily refactor your current system is the way to go, but even then you will sooner or later run into problems your original design did not consider.
In the end it mostly comes down to reducing complexity, but the goal is always allowing new features to be added as fast/easy as possible.
Because I'm lazy.
quick hacking together of prototype systems delivers a dopamine high that is quickly reduced to zero when those systems need to be maintained and extended into an indefinite future
building systems that are well structured and not terrible requires knowledge and experience, but absolutely 100% does not take more time than building shitty hackathon prototypes