In the same way that we try to use semantic versioning to prevent unintended breakages in dependents of our public-facing APIs, I think we developers should start considering the UI/UX of our apps to be their public-facing API, and use the philosophy of semantic versioning accordingly.
When we rename or remove a function from an API, that's a breaking change, any dependent software will no longer work unless it's modified to take that change into account.
Similarly, when we move, rename, or remove a UI element, keyboard shortcut, or visual affordance, that should be considered a "breaking change" for our dependents, the humans on the other side of the screen. And in the same way, we should avoid making such changes unless the long-term benefit of doing so outweighs the short-term cost. Plus, users will know that moving from 7.x.x of your software to 8.x.x will require them to relearn some aspect of it.