Hacker News Reader

Comment by cschmidt | Hacker News Reader

Since the OP is prompted by Judea Pearl's new book, I'll ask here. There seem to be at least two schools of thought in causal statistics. The first is championed by Judea Pearl [1,2,3] and the other by Donald Rubin [4].

If I want to learn causal statistics, for use in ML, which school of thought would be more useful? I don't mean to prompt any causal flame wars, but it isn't obvious which approach is more useful.

    [1] https://www.amazon.com/Book-Why-Science-Cause-Effect/dp/046509760X
    [2] https://www.amazon.com/Causality-Reasoning-Inference-Judea-Pearl/dp/052189560X
    [3] https://www.amazon.com/Causal-Inference-Statistics-Judea-Pearl/dp/1119186846

    [4] https://www.amazon.com/Causal-Inference-Statistics-Biomedical-Sciences/dp/0521885884

View on HN

I studied Pearl and Rubin in college and didn't find them to be particularly antagonistic, though indeed they use somewhat different terminology and emphasize different things.

If you'd like to learn more, just use https://www.hsph.harvard.edu/miguel-hernan/causal-inference-.... It's an awesome resource and free.

Start with Rubin. Potential outcomes are more grounded and concretely interpretable. The epistemology is simple and won't get in your way, if you're inclined to dwell on those things. It also has the huge wealth of well developed techniques. Then move on to Pearl.

(btw this is in no way a "standardized" answer, just my opinion about how to start with the basics)

If there is more than one "school of thought," then there cannot be any real difference[0] between their results: because if there was, it would count as an empirical resolution between them. Everything else is metaphysical paint and should be taken according to taste.

[0] In saying this I'm assuming neither one is outright wrong.

But one way can be vastly easier to get results than another other way. Even easy but wrong ways can be useful, such as Newtonian mechanics. (It's computationally infeasible to predict, eg, robot movements with quantum mechanics.)

Newtonian and quantum mechanics deliver different answers, which means that without approximation they are different theories - not different schools of thought. With approximation (in a certain limit), quantum and Newtonian mechanics are the same, and take the same amount of work to find answers - quantum mechanics yields Newtonian mechanics in that limit, doing "room temperature large scale" quantum mechanics means writing down Newton's laws. In this sense it's most accurate to say that Newtonian mechanics is a particular approximation to quantum mechanics that holds under certain circumstances.

This other school-of-thought business (Bayesians vs. frequentists, pilot waves vs many worlds vs Copenhagen, everything like them) means choosing between different sets of words to describe the same thing. You are left with one of two cases: either they can be shown equivalent (in which case people are prone to keep on arguing over which one is better), or they are different (in which case one is wrong.)

There is no upper or lower bound on the minimum length of a proof when you are free to choose the language in which it is expressed. This means that there almost certainly are things can only be communicated in one language but not another, even if the languages are equivalent in some vague limiting sense.

So as a practical matter, you have to work in a better language, because the difference between spending 30 years to learn something and 30,000 years is quite significant.

Just to throw in a metaphor, rather that Newtionian and Quantum mechanics, perhaps the schools of thought are like the differing formulations of classical mechanics like Newton Vs Hamiltonian Vs Lagrangian formulations.

Are you familiar with Thomas Kuhn? I ask because he wrote a book basically arguing that's not how science works at all.

Because the total amount of phenomenon explainable is too large for any theory, you cannot test or even hold entire concepts of "what each theory says" in your mind.

Schools of science form into communities which determine what is deemed "in scope", what are the grounding frameworks and concepts of the theories that explain the foundational phenomenon really well, informally what is out of scope, and the border territory of active research anomalies.

I only bring it up because I think it's one of the few works of genius and insight into how science and knowledge actually works in practice.

>Because the total amount of phenomenon explainable is too large for any theory, you cannot test or even hold entire concepts of "what each theory says" in your mind.

At the far end of "models work very well and the epistemology is solid," physics solves this problem by assigning different people to each phenomenological class (organized by the engineering similarity of the experimental devices needed to probe them) and then using math to check that everyone's individual confirmation of the theory in their area fits in correctly to the bigger picture. As a result even though the frontier of physics is too large for any one person to know, the confirmation of the standard model has been built into an unbroken surface that reaches all the way from the highest energies achieved to chemistry and astronomy.

When you have a theory like that, you can prove mathematically that it is equivalent to other theories. Then, the enlightened can stop arguing about which one is "truer!" If you have two theories that only exist in the form of English sentences (this was true of psychology in Freud's era) I can't imagine what an equivalence proof would look like, even if it would be possible. Fields where you can't formalize anything tend to have philosophies that look more and more like critical theory as you move further and further from pure logic. At the far end, the empiricism is completely phenomenological and the theory is nothing but literature with no predictive power (and as a result, the only way to choose between them is by disguising aesthetic arguments as appeals to this-or-that). I can't think of any fields that didn't look like that in their infancy, and success has usually been associated with progress away from that.

> physics solves this problem by assigning different people to each phenomenological class (organized by the engineering similarity of the experimental devices needed to probe them)

That's an example of how multiple equivalent formulations of the same theory can be useful for different things even though they are ultimately equivalent.

Reply on news.ycombinator.com