I have experienced the same thing myself, but I would explain it differently.
Even in science where the focus is actually on proving things, it is difficult to prove causation. I see this more as a question of whether you act on a correlation that you’ve discovered or whether you simply talk about it.
While it’s fine to discuss things, it sounds like at the small company you worked at, that that was more important than taking action (or perhaps the company was too insecure to make a decision about what to do).
At the big company, the focus was on making experiments and seeing what the results are, and then iterating on that. In the end, the business doesn’t care whether something causes something else, it is only interested in adjusting things, measuring results and if they increase some important metrics, then the business is happy.
That said, it can be taken to an extreme where only the metrics are thought of as important, rather than other qualitative or hard-to-measure things.
It is much easier with web companies who can quickly roll out changes and use AB testing to run experiments. Also, those companies with larger user bases will have better sets of data on which to base their decisions. Companies that are well established will also have historical data that will make it easier to interpret results.
It also sounds like at the first company, the tech team was interested in being theoretically correct, while at the second one, the engineers were interested in trying things.
It is an understandable problem in some ways: good programmers are lazy and won’t want to make unnecessary changes, so they tend to argue against changes that haven’t been proven ahead of time to yield results. However, for the business to thrive, knobs must be adjusted and experiments made. And those programmers have to turn the knobs (or make them adjustable by someone else). So there is a strong counter-motive at play there.
However, good programmers who are also engineers with a focus on the product / business will understand the importance of doing this. It’s important for the engineering team to be aligned with the business and it helps to have cross functional teams. Otherwise you may end up with departments protecting their own interests. See also Conway’s law and its derivatives.