Some of the most important deep learning papers | Hacker News Reader

Some of the most important deep learning papers

adeshpande3.github.io

308 pointsadeshpande15 comments

Open articleView on HN

With this field advancing so fast, I guess if we could do something like this, that would be great:

Maintain a running list of:

- 3-5 most important papers in the last 3 months

- 3-5 in the last 1 year (not all in the 3 month list would make into this list)

- 3-5 in the last 5 years.

I guess it's difficult for a small number of people to rank the papers. Maybe a hackernews or reddit style upvote/downvote system can be used, with a list that essentially scrapes arxiv for papers.

> I guess it's difficult for a small number of people to rank the papers.

This is what's done every year at the AI conferences. No need for a new voting system.

- 3-5 most important papers in the last 3 months: The "best paper award" deep learning papers of the most recent 1-2 AI conferences.

- 3-5 in the last 1 year: Top cited deep learning papers from AI conferences this year.

- 3-5 in the last 5 years: Top cited deep learning papers from AI conferences in the last 5 years.

Since AI conferences don't happen every day (or even every month) this would not be a hard list for a novice to maintain.

>This is what's done every year at the AI conferences. No need for a new voting system.

Not really. Academia rarely ranks things important to the application (aka industry) of a field of study. An efficiency improvement that completely changes the economics of something would likely be rejected from an academic conference for being 'incremental'. Similarly, 'novel' work is much more highly rewarded in these conferences than improvements to existing techniques, experiment replications, or negative results, all of which are more important to industry.

Bibliometrics are garbage and represent little more than a popularity contest (and what is controversial). It's basically like if you took up votes and down votes on reddit and just had them both increment a single score. If they were worth anything, there wouldn't be such a thing as review papers that highlight important developments.

> Academia rarely ranks things important to the application (aka industry) of a field of study.

"Rarely" is a strong word. I think the truth is closer to, "Academia has screwed up as a whole in a very small number of cases, but usually works just fine, and always self-corrects eventually."

I'm not talking about screwups. I'm talking about fundamentally unaligned objectives between academia (advancing the edges of knowledge) and industry (making knowledge useful in an economic sense).

It's not a problem, it's just something you have to recognize so you don't have the wrong expectation.

If only we had a way of learning how to rank these papers...

Good list! I think it's important to note that this article is (intentionally) focused on modern CNN architectures, and not "deep learning" in general.

I'd also add in the following "technique" articles: Geoff Hinton et al.'s dropout paper[0] and Loffe and Szegedy's Batch Normalization paper[1]. I don't think there's been enough time for the dust to settle, but I'm excited about the possibilities Stochastic Depth[2] could offer, too.

[0]: http://arxiv.org/abs/1207.0580 [1]: http://arxiv.org/abs/1502.03167 [2]: http://arxiv.org/abs/1603.09382

We've compiled a fairly extensive list of deep learning papers here: http://deeplearning4j.org/deeplearningpapers

We also Tweet out new ones as they're published here: https://twitter.com/deeplearning4j

If you had to pick one which would it be? the oldest one?

I really enjoyed this write up .. would be so great if research papers in general were commented on like this (what is interesting, what is the significance of the result etc)

Here is the same list but including another important paper, inception-v4, it beat MS resnets on top 5 error.

https://github.com/andrewt3000/MachineLearning/blob/master/c...

Glad to see R-CNN and its follow-on work on the list. We've been using R-CNN for a few weeks now and have seen great results on object detection and localization. A few papers this year have played around with substituting different convnets and different classification schemes and improving the network in various ways. I'm excited to see where this specific architecture goes in the next few years.

Yes, it's a lot of papers, this might help people keep on top of the most important ones (though it works from the top of the queue, and not from the bottom up) http://www.arxiv-sanity.com/

Only applied to image recognition.

The "Generating Image Descriptions (2014)" paper also involves text.

Reply on news.ycombinator.com