Hacker News Reader

Comment by free_rms | Hacker News Reader

Yes, a linear regression is explainable. You know what your features represent, and the regression puts a straight-up co-efficient on each of them.

You can look at the weights you trained, see how strongly your input features contribute to the output prediction, and say things like "if all else is equal, 25-49 males are 0.08 likely to click on this ad".

Deep nets aren't explainable in that way, even if you devoted a lot of effort. Maybe someone will come up with a novel method for doing it but as far as I know the best we can do is statistics about what the net's likely to predict.

View on HN

> Yes, a linear regression is explainable.

I don't agree that this is, in general, the case.

Dense linear models trained on a lot of parameters aren't exactly simple to trace, and even the weights aren't as intuitive as they might seem.What about regularization? What about pre-processing? What about feature selection?

There are a lot of examples of linear regressions where, for example, you can reverse the sign of the correlation by picking some features rather than others.

Assigning meanings to features of your model is something that requires extreme care and deep understanding of both the model and your dataset.

Ok, fair enough, the more sophistication you add, the less explainable it gets.

That being said, you can look at magnitude and direction of various coefficients and conclude certain obvious things. Even the case where reversing the correlation by picking some features instead of others says something -- those features point different ways but one has a higher magnitude so it 'wins'. They're ultimately pretty simple statistical observations about the training set.

A deep net, in comparison, is a totally unexplainable black box between the cross-linking and the sigmoid functions.

I agree that there is an intuitive concept of "explainability", which is why I said that a formal model for it would be interesting to see. I also agree that "the average explainability of a deep net is lower than the average explainability of a linear model" (whatever that sentence means), my point is that it's extremely hard to pinpoint what explainability means, because the usual characterization of "you can fit the model in your head" is extremely subjective and brittle.

I'm all for more research in this field. Heck, I'm all for more research in any field, but because our definition of "explainability" is so primitive and unrefined, arguing that we should build ethical or regulatory frameworks on top of it seems premature.

> you can look at magnitude and direction of various coefficients and conclude certain obvious things.

Sometimes, but if you have correlated predictors, the 'explainability' can be distributed across coefficients of those predictors in unpredictable ways. Training a linear model twice with two different subsets of training data can give you very different coefficient weights.

Even linear regression is not very easily explainable for real world systems with currently utilized technologies -- especially when you have a virtually unlimited range of mechanisms intentionally or unintentionally subset or modify the sampling process (or select for desirable results at "sampling design time") ...

Reply on news.ycombinator.com