Hacker News Reader

Comment by knzhou | Hacker News Reader

The article calls machine learning "conservative" because it only tells us what is, and not what should be. I don't think that's a useful framing. It's more accurate to say that, like all statistical techniques, machine learning is descriptive, not prescriptive. Not everything in the world has to be prescriptive.

View on HN

The article is not addressing what machine learning can and cannot do but how individuals and organizations conceptualize what machine learning can and cannot do.

It, as the current top comment mentions, is a philosophical rather than technical argument.

It doesn't matter if all machine learning can do is describe the current state of things of it is presumed by those interpreting the output to provide insight into some great and universal truth.

Most science degree programs have a philosophy of science course as a degree requirement. As an engineering faculty member I have found huge resistance to and equivalent course becoming part of engineering or computer science degrees. Those degrees, instead, take an apparoch to scientific/technical knowledge that is less science and more religion.

The comments here are a reminder why such courses have a place.

It is not just a philosophy course which is missing from software engineering degrees. Software has real impact on individuals and societies that use them. Therefore, software engineers must necessarily be taught about the interface of psychology/sociology with software and the ethics surrounding this interface. The world would be a better place for it.

Please read this thread https://twitter.com/yonatanzunger/status/975545527973462016 "computer science is a field which hasn't yet encountered consequences."

> Not everything in the world has to be prescriptive.

Unfortunately we're still on the rise of the current hype cycle around Machine Learning, and prescriptive solutions are precisely what nearly every company who use the phrase "machine learning" are promising.

I think recasting prescription from "so you should" to "and here are the inferences now available" can be reasonable. The issue is when the inferences from discovered patterns are invalid and implicit - as per predictive policing or follow about ads on the web. An inference such as "this is almost certainly the component that is causing the most faults and will cost the least to replace" is reasonable and valid to me, and so a machine offering it would be valuable.

Maybe the pure science of ML is descriptive and not prescriptive. But if the rest of the world uses ML prescriptively in application, then in practice it's basically prescriptive.

It's a bit like dictionaries: they "learn" the current meaning of words. It gets annoying though when everyone starts running around policing the meaning of words based heavily on that dictionary: it would conservatively restrict the natural evolution of word meanings, when we know meaning of words do change over time, and sometimes should change over time.

We generally want ML to imitate us (or human generated data), thus we teach it to do so. But there are also generative ML (like GANs) and simulation based ML (like AlphaGo) which can be more creative. There is nothing stoping us from letting agents evolve in a complex environment and be creative. It's just not commercially useful to do that yet. Doctorow writes like he doesn't understand much math behind ML, yet has strong opinions on it.

Every time a random number is involved in the training process (stochastic selection of mini batches, noise injection, epsilon-greedy action selection), a form of exploration is taking place, finding novel paths of solving the problem at hand. The mixing of noise with data is paradoxically useful, even necessary in learning.

Sure, but no matter what, you can't derive an "ought" from an "is". At the end of the day we merely tell these algorithms what is the case. No matter what goes on inside, they cannot output moral prescriptions.

>Sure, but no matter what, you can't derive an "ought" from an "is".

The same applies to humans, no?

No it doesn't. When a humam gets wronged by what "is" they can likely feel or imagine a better "ought".

They can feel or imagine an "ought", but the point of Hume is that they can't argue for it.

And yet humans do, all the time. It's similar to Hume's attack on causality. You can't show that A caused B, but yet we all act like it's the case, when B always follows A. Kant's critique comes next.

You don’t think an machines can learn to classify situations as good or bad for them ?

> ... finding novel paths of solving the problem at hand ...

This only applies to reinforcement learning (and similar). Classification problems (and similar) are fundamentally conservative, because you feed them with a finite set of preexisting training data that they must extrapolate from.

> It's just not commercially useful to do that yet.

Some people are working on it : https://news.ycombinator.com/item?id=21728776 (dec 2019)

Reply on news.ycombinator.com