"that would mean that KPIs shouldn't be the sole measure of our performance that that doesn't make sense!"
My experience in the field has been that an astounding number of products have been destroyed and users harmed by failing to heed Goodhart's Law.
Edit: but that person is unlikely to be subject to KPIs themselves.
For example, if I'm a manager that can make my people hit their KPIs, and my KPIs are about getting my people to hit theirs, then I'm subject to KPIs, and I like it. It's easier than making my people succeed at what really matters, and it makes me look good.
When KPIs are explicit everyone knows what they are and can modify their behavior to optimize for their KPIs when all measurement goes away the new KPI is the arbitrary one held in the decision makers head, and now instead of it being an explicit bar that can be objectively used to make decisions the entire system falls apart into politics and emphasizing appearances over work because the only thing that matters with implicit KPIs are what everyone else thinks of you, which is much easier to manipulate than the amount of cash you brought in.
You can reduce a fever by treating the underlying infection, soaking in ice water, or taking acetaminophen. No good doctor would judge a patient's health solely because a single metric, namely body temperature, was able to be moved into acceptable ranges. That doesn't mean temperature isn't extremely valuable data, and essential to decision making, but that it cannot be a substitute for understanding and solving the real problem.
I once knew of a SaaS company that had perpetually growing MRR (Monthly Recurring Revenue), great right? Except, churn was also growing. An increase in MRR was achieved by upselling a perpetually shrinking group of core customers. The core KPI of this company was MRR, and, unsurprisingly, this company does not exist anymore. Again, here is a case where we can see all that other data (churn, upselling) is very useful, as is the KPI. But the key to success or failure here is whether or not you want to really expend the effort to understand the problem or just chase a KPI.
KPIs are seductive because they make managing team's performance seem much easier: just get this number higher and you're doing good, get it lower and you're doing bad. But that's like playing a game of chess where each piece is controlled by a different person, and that person is judged solely on how many times they can get the king in check.
That's a good example because as Strathern's formulation notes, the problem lies in make the metric the target. It would be folly to think that reducing a patient's body temperature to the normal range is sufficient for curing illness. GE's Jack Welch famously focused solely on the stock performance as a measure of success. It worked, by that measure GE was wildly successful. By almost any other measure Welch destroyed GE https://www.bnnbloomberg.ca/jack-welch-inflicted-great-damag...
/s
Like I'd agree that maths/cs people are little naive about error but I can't imagine any data scientist from a social science background thinking that any measure is perfect.
Wow, so when I worked at a FAANG, I would say that soc sci people (broadly defined) made up at least a third of the data science org. Data science is a broad church though, and varies a bunch across companies so it's normal I supppose (though strange to me).