The biggest mistake is engineers owning experimentation. They should be owned by data scientists.
Realize though that is a luxury, but I also see this trend in blue chip companies
Realize though that is a luxury, but I also see this trend in blue chip companies
As a data scientist at a "blue chip company", my team owns experimentation, but that doesn't mean we run all the experiments. Our role is to create guidelines, processes, and tooling so that engineers can run their own experiments independently most of the time. Part of that is also helping engineers recognize when they're dealing with a difficult/complex/unusual case where they should bring DS in for more bespoke hands-on support. We probably only look at <10% of experiments (either in the setup or results phase or both), because engineers/PMs are able to set up, run, and draw conclusions from most of the experiments without needing us.