Hacker News Reader

Comment by qsort | Hacker News Reader

Playing the devil's advocate here, I'm with you that half of AI ethics is obvious and the other half is wrong, but is't it the goal of the field to try and give meaning to things that aren't obviously well defined?

To make an example that's en vogue right now, AI explainability. Nobody even has a definition of what it means for a model to be explainable (is a linear regression "more explainable" than ML? isn't Google search far less explainable than any model of anything ever?), but a reasonable framework for that concept could certainly be interesting.

Obviously, serious frameworks are done with definitions and math, not with words and storytelling, but all the air around those things seems to me more the fault of politicians, crap journalists and freaking idiots on social media (the current term is 'influencer', I reckon) rather than an issue with the field itself.

View on HN

Sorry to ask, but I don't understand the points you are making. Could you please elaborate on them? I had trouble with the last 2 paragraphs. Thanks

AI explainability is a concept that's being thrown around frequently these days; it revolves around the idea that machine learning models should be "explainable", that is, their predictions should be traceable not just to the mathematical operations that define it, but also to some properties of the input which should be understandable by a human.

While I won't deny that the concept is interesting, it's terribly difficult to understand what it would translate to, technically speaking. It's true that machine learning models make predictions that are hard to check (and sometimes even understand), but they aren't inherently "less explainable" than even the simplest statistical models, like linear regressions. For example, it's pretty weird to be angry that "ML is not explainable" but to be okay with things like Google search that have literally zero transparency.

My main problem with it is that people with poor understanding of computer science and math in general - let alone machine learning - throw around the term like it's obvious what they mean, when their real goal is generating social media clout in the case of journalists and influencers, or, more darkly, to enforce political control of the industry in the case of politicians.

I think it's a little better than that.

An example from class. Suppose you are building a ML decider which can stop a production line if it sees defective products. If you are choosing between a decision tree and a neural net, one of the things to consider is that with a DT you can look at the tree the model comes up with a d say, okay if the mass of the widget is low, we reject.

With a NN, you can't see why things are rejected in the same way.

Some tasks benefit from having more explainable models, some it doesn't matter. But I don't think it's just a buzzword or trying to enforce political control.

> But I don't think it's just a buzzword

I don't think it's just a buzzword either, in the same sense that 'AI' is not just a buzzword. But both "AI" and "explainable" are also buzzwords, or at least often used as such.

I have no objections to the example you made, some models are "obviously" more explainable than others. I'm simply refuting the claim that some models are inherently more explainable, because the entire concept starts falling apart when you take it out of its mathematical context. For example, it's easy to see what a DT does when it's small, but larger DTs are as "unexplainable" as a NN.

> trying to enforce political control

I'm not against political control per se, some political control is well-justified; but I find it pretty suspicious that everyone is jumping on this train when there are more glaring issues, (for example, if and when a branch of the government or a government-controlled agency decides to use a ML model, they should be required to declare that they are using it and make the source code public, so that it can be cross-examined) and my guess is that it's because "explainability" provides a nice narrative, unlike other concerns.

I think linear regressions are perfectly explainable.

https://en.wikipedia.org/wiki/Proofs_involving_ordinary_leas...

Yes, a linear regression is explainable. You know what your features represent, and the regression puts a straight-up co-efficient on each of them.

You can look at the weights you trained, see how strongly your input features contribute to the output prediction, and say things like "if all else is equal, 25-49 males are 0.08 likely to click on this ad".

Deep nets aren't explainable in that way, even if you devoted a lot of effort. Maybe someone will come up with a novel method for doing it but as far as I know the best we can do is statistics about what the net's likely to predict.

> Yes, a linear regression is explainable.

I don't agree that this is, in general, the case.

Dense linear models trained on a lot of parameters aren't exactly simple to trace, and even the weights aren't as intuitive as they might seem.What about regularization? What about pre-processing? What about feature selection?

There are a lot of examples of linear regressions where, for example, you can reverse the sign of the correlation by picking some features rather than others.

Assigning meanings to features of your model is something that requires extreme care and deep understanding of both the model and your dataset.

Ok, fair enough, the more sophistication you add, the less explainable it gets.

That being said, you can look at magnitude and direction of various coefficients and conclude certain obvious things. Even the case where reversing the correlation by picking some features instead of others says something -- those features point different ways but one has a higher magnitude so it 'wins'. They're ultimately pretty simple statistical observations about the training set.

A deep net, in comparison, is a totally unexplainable black box between the cross-linking and the sigmoid functions.

I agree that there is an intuitive concept of "explainability", which is why I said that a formal model for it would be interesting to see. I also agree that "the average explainability of a deep net is lower than the average explainability of a linear model" (whatever that sentence means), my point is that it's extremely hard to pinpoint what explainability means, because the usual characterization of "you can fit the model in your head" is extremely subjective and brittle.

I'm all for more research in this field. Heck, I'm all for more research in any field, but because our definition of "explainability" is so primitive and unrefined, arguing that we should build ethical or regulatory frameworks on top of it seems premature.

> you can look at magnitude and direction of various coefficients and conclude certain obvious things.

Sometimes, but if you have correlated predictors, the 'explainability' can be distributed across coefficients of those predictors in unpredictable ways. Training a linear model twice with two different subsets of training data can give you very different coefficient weights.

Even linear regression is not very easily explainable for real world systems with currently utilized technologies -- especially when you have a virtually unlimited range of mechanisms intentionally or unintentionally subset or modify the sampling process (or select for desirable results at "sampling design time") ...

Reply on news.ycombinator.com