Machine learning is by definition a black box. Nonlinear statistical models with huge numbers of parameters to be estimated.
But explainability at a deeper level is also critical towards developers, to close the feedback loop and iterate towards better models.