When you try to look backwards and say "this is why it works", that's not a lie, but applying the learnings in a forward moving sense are filled with exceptions and traps. So you're kinda back where you started, where the palette of things to try gives you inspiration, and then again, you gotta put it to users.
Anyway, great for UX people that they command so much money, but I think that's less that they are objective and scientifically based and more because they are good at navigating the politics of decision making.