Yann LeCun Deep Learning Course 2021
atcold.github.io
atcold.github.io
The site itself appears to be additive (only new materials posted in later weeks) to the old 2020 site, which is here: https://atcold.github.io/pytorch-Deep-Learning/
I thought I was going to repropose the same practica I've used during NYU-DLSP20, last year edition, just in different order.
But I couldn't.
This year's students have LV-EBMs on their side. We told them about the cake and now I cannot pretend it doesn't exist and teach as if they were unaware of the elephant in the room. It would have been intellectually dishonest. Henceforth, I've redesigned my whole deck of slides.
...
Last year material is still valid. This year you have a different take. A more powerful one.
A lot more examples on SSL too.
Alfredo also went all in on the visuals again. Amazing job.
I haven't found any CS curriculum that teaches causal inference in any way (maybe in optional courses, but that misses the point) and I think that it is a huge mistake. Causality is more important than simple prediction.
I don't understand why CS people escape causality in most courses, not even mentioning it.
> We must keep in mind the lessons: Inferring cause and making predictions are different tasks.
Models that are causally incorrect can make better predictions than those that are causally correct. Yes, it's a paradox.
(These titles are from memory bc I am typing with one hand as I hold a sleeping infant with the other.)
- taddy, et al “deep IV”.
- Greg Lewis at Microsoft research has work here too... the title of that paper I cannot remember.
- Farrell, Liang and Misra, “neural networks for estimation and inference” (econometrica 2021)
- For non neural network based ml informed approaches to causal inference, victor chernozhukov, alexandre Belloni and Christian Hansen have a long series of papers going back to 2011 (sometimes with other coauthors) which are generally based on the LASSO in settings with many instrumental variables.
- chernozhukov, demirer, duflo, et al have a paper on “double machine learning” which is relevant. I think there is relevant work in biostat by Jamie Robins, but that is not my field.
- athey, imbens and wager have papers on random forest based approaches - at least one is in JASA. Might be called “causal forests” ?
- since economists frequently estimate in the GMM (generalIzed method of moments, not Gaussian mixture model) framework, there is some recent work by... Kallus (?) on formulating GMM as an adversarial game. Greg lewis and V. Syrgkanis have also worked on this problem but the title of the paper escapes me.
I would like to teach this material myself so there will one day be a class (inshallah) but so far have not had time to organize it yet!
If you every make a class out of it, please post it here!