"Should" in an ideal world doesn't equal "should" in a world with deadlines and limited resources. It's enough work to ensure that all the features work with ONE version of an app. If you had to also test every version of the app without this or that feature update installed, you just continually multiply the testing requirements by the number of optional features.
And don't say unit testing solves this problem. The problem is in INTEGRATION testing with the whole app, which 99% of the time can only adequately be done by making the build and having a human use it. Even automated integration tools won't find things like formatting errors and buttons too small for humans to click.
I'd like to continue to maintain ONE app, please, and not `n!` app permutations.