It's not worth it to gain 30% more customers by spending 10x the original development time micro-optimizing everything.
The issue is that so many projects have not been designed with these ideas from the outset, that it indeed usually does take a mammoth amount of re-engineering and dev time to make these kinds of changes and optimizations after the fact.
To continue with the mediocre carpentry analogies floating around: a (competent) carpenter doesn't build a sturdy table by first building a shitty table and reinforcing it later.