Seven Pillars of Causal Reasoning with Reflections on Machine Learning [pdf] | Hacker News Reader

Seven Pillars of Causal Reasoning with Reflections on Machine Learning [pdf]

ftp.cs.ucla.edu

158 pointsdiego89812 comments

Pearl's paper is a solid generalization on logical reasoning and intelligence, but we have to be wary that even us, humans, aren't perfect at generalization and regularization. Without going too meta, we have to contemplate whether logical reasoning causes human level intelligence or it's a mere effect of a complex statistical phenomenon. It may be the case that a human-level AI system will contain all seven pillars of causal reason, but it does not mean that these reasoning were specifically programmed into the AI.

A similar example of this human over-regularization can be found speech technology, linguists have generalized 50~60 phonemes in English speech, though in real life speech, there are magnitudes more phonemes in play, if we force our speech recognition systems to classify speech into 60 phonemes, we miss out on important details. This is why end to end systems such as WaveNet is so much better than older systems, because when we ditch our over-regularized concept, it can learn more freely.

> we have to contemplate whether logical reasoning causes human level intelligence

Human level _logical_ reasoning has not existed for long time and can't be the cause of human level intelligence because it is very little used in practice. Natural logic has existed little over 2000 years among subset of intellectuals as a method for reasoning.

Natural reasoning is one of the causes of human level intelligence It's basically just a bag of heuristic rules used in qualitative, action-change and taxonomical reasoning. There is no evidence that deduction has significant role in commonsense reasoning.

> There is no evidence that deduction has significant role in commonsense reasoning.

As a matter of fact the first logical system heavily used induction, not the duction, to extend regularity over unknown topics instead of exploring the unknowns systematically and deriving the regularities out of the experiments.

I'm surprised Pearl doesn't mention reinforcement learning once in this essay. This domain is where Pearl-style causal reasoning frameworks most closely meets data-driven statistical machine learning methods. Methods such have Dyna (link below) can be seen as reinforcement learning with some degree of "reasoning" about future consequences of actions. I suspect that there is more work to be done in embedding causal methods into planning, though it's pretty clear that a lot of progress is being made in domains such as Atari.

Dyna: https://www.cs.cmu.edu/afs/cs/project/jair/pub/volume4/kaelb...

I agree. RL is most definitely a form of causal reasoning in my book. It involves associations(value iteration), intervention(action/reward) and counterfactual (policy iteration) that Pearl suggested. I think Pearl did not mention specific technology as it's more of a meta discussion.

You may be interested in this talk from one of his collaborators (I’ve not watched it):

https://vimeo.com/238266313

Thanks for this.

My intuition is that a "program induction" approach has the potential make causal learning possible (without having to predefine the causal graph as I believe Pearl requires in his techniques). Here is a description of my failed attempt:

https://www.quora.com/What-deep-learning-ideas-have-you-trie...

Wow, how the field advances in but a year!

You should take a look at Pyro, and maybe try out implementing some form of variational guide for an adaptor-grammar model of programs. There are papers already claiming to have done it, just not for program induction.

I have a few ideas and intuitions to add to yours, so email me via my profile if you want to collaborate.

I wasn't aware of Pyro. Thanks for the reference. I might be interested in collaboration one day, however I would be a terrible collaborator right now as I can only find sporadic short slices of time to put in these sorts of projects. It's rare people show interest in my work so I'll keep you in mind if ever I stumble upon something interesting enough to be worth spending collective efforts on.

Interesting. I wonder as a next step if you could use a causal graph as part of the controller for producing the programs. So this graph would be predefined but it would able to "reason" on the generated programs. That reasoning could be based on program complexity/minimum description length. So your controller is a supervisor that takes input/output and generates programs that recognize circles.

What kind of reasoning allowed Kepler/Newton/Einstein to derive the laws of gravitation? Was it causal reasoning? Logical reasoning? Does Kepler's reasoning belong to the same "category" as Newton's? Einstein's? Not sure this question can even be formalized. Godel theorem seems to show it can't.

Reply on news.ycombinator.com