I am not convinced the explanations we’d get from these models would be interpretable enough to drive their conceptual improvement. Deep nets just don’t learn nice interpretable features, nothing in the training objective makes them do so. E.g. they make use of really dubious features [1], and might not actually learn “hierarchies” of features as we thought they were [2].

[1] https://arxiv.org/abs/1905.02175

[2] https://arxiv.org/abs/1904.00760