> But mostly I think the effort towards explainability is fundamentally misguided... explainability is a mysterious unmeasurable goal.
Do you think people should stop working on it?
Most interesting things seem difficult to measure, until someone finds a way, and then it seems obvious. An example is search engine quality. At first this might seem too subjective to be measurable. But Google started measuring search quality using a panel of humans, and now everybody does that.
The whole idea of these challenges is to broaden the search, to hope for key insights that by their nature seem elusive at first.