Practically all web marketing experiments and analysis is invalid, and not replicable. But it pays the bills for everyone to pretend its legitimate.
The worst part is that some of the tests were still used to justify decisions to and by management. SMH
It’s a global optimization problem, not a scientific way to understand how a specific change impacts users. Testers that have this mindset tend to be locally constrained and less likely to have bigger wins.
No. It's the fault of middle management.
If you are unable to provide your manager reasons for new development, he's going to find someone else to do your job, someone who will give him a report with charts and numbers that he can use to justify and expand his teams operational and headcount budget.
Giving a manager a report that says "there are little to no modifications that we can make at this time to improve UX" is a CLM for him.