Comment by YeGoblynQueenne | Hacker News Reader

Comment by YeGoblynQueenne | Hacker News Reader

YeGoblynQueenne

>> To appreciate the extent of this denial, readers would be stunned to know that only a few decades ago scientists were unable to write down a mathematical equation for the obvious fact that “mud does not cause rain.” Even today, only the top echelon of the scientific community can write such an equation and formally distinguish “mud causes rain” from “rain causes mud.” And you would probably be even more surprised to discover that your favorite college professor is not among them.

I know who Judea Pearl is, but this is just conceited. "Only the top echelon of the scientific community"? What, of _any_ field?

More importantly, the scientific community can express "mud does not cause rain" in a formal manner and they have been able to do this since the beginning of the last century:

  ¬Causes(mud, rain) ∧ Causes(rain, mud)

It's not specifically an equation -it's a theory- but it does the job in a formal language with Turing equivalent expressive power. So does this:

  p(mud | rain) = 1.0
  p(rain | mud) = p(rain)

Additionally, in modern machine learnign there are theoretical results that prove the learnability of all computable functions by various different classes of algorithm -inverse deduction for symbolists (like myself), backpropagation for the connectionists and so on [1]. It's really hard to reconcile such theoretical results with the claim that there are, basically, things you can't learn with, say, neural networks or genetic algorithms, but can learn with counterfactuals.

_____________

[1] Nice little talk by Pedro Domingos summarising his "Master Algorithm" ideas, including a bit about Judea Pearl's favourite type of machine learning:

https://www.youtube.com/watch?v=B8J4uefCQMc

  >  p(mud | rain) = 1.0
  >  p(rain | mud) = p(rain)

I'm not sure these statements say what you think they do. Assuming P(rain) > 0, the first statement

  p(mud | rain) = 1.0

is equivalent to

  p(mud & rain) = p(rain).

The second statement therefore implies

  p(rain | mud) = p(mud & rain).

Thus

  p(rain & mud) / p(mud) = p(mud & rain).

This, however, leads to the surprising conclusion

  p(mud) = 1.0.

According to your theory, it is always muddy.

I think what's meant is something like p(mud_{t+1} | rain_{t}) = 1 and p(rain_{t+1} | mud_{t}) = p(rain). Say during 10 units of time it rains at points 1, 3, 4, 6, and 10 and it's muddy IFF it rained in the previous period. Then the above probability statements are true and it's only muddy 40% of the time.

OK, but that's not a big problem. My theory only needs to say that mud doesn't cause rain. Judea Pearl says that only his addition to Bayesian probabilities can do that.

I guess I could have thought of a better example, but I like how ryanmonroe's comment interprets it, below.

> p(mud | rain) = 1.0

> p(rain | mud) = p(rain)

tr352 has already detailed below why this doesn't work, but I think it's worth noting why this is fundamentally the wrong thing. When you look at P(X|Y), you're conditioning on everything you can infer having seen Y -- you're not limiting yourself to causal connections. So in fact, you're not correct to say that P(rain|mud) = P(rain); P(rain|mud) will be greater than P(rain); the fact that it's muddy out does indeed tell you that it is more likely to be raining out, even though the causality does not run that way.

(Indeed, in general, P(X|Y)>P(X) if and only if P(Y|X)>P(Y) -- this occurs if and only if X and Y are positively correlated -- but causality is not symmetric, so you certainly cannot represent causality with "being positively correlated", a symmetric relation! Similarly P(X|Y)=P(X) if and only if P(Y|X)=P(Y); this occurs if and only if X and Y are independent. I'm pretty sure you don't mean to say that rain and mud are independent of one another, but that's what we must conclude from your equations.)

Basically, you're talking about P(rain|mud), but what you actually want is something like P(rain|do(mud)); it is indeed the case that P(rain|do(mud))=P(rain). But pure probability theory can't capture that, which is the point of what Pearl is saying. Probability theory really does not work like causality works, and cannot capture the same information, and if you ignore this and try to use probability theory as if it were causality anyway you are going to end up with some seriously nonsensical results, like thinking that rain and mud are independent of one another.

(Meanwhile, the problem with your first attempt, namely,

> ¬Causes(mud, rain) ∧ Causes(rain, mud)

is, well... it's not wrong, it's just completely unhelpful. OK, so you've made "causes" into a primitive relation. Great. Now how do we reason about that? All you've done is rephrased a statement, what we need here is a system of reasoning.)

Reply on news.ycombinator.com