Comment by karpathy | Hacker News Reader

Comment by karpathy | Hacker News Reader

The fact that RCNN does a wasteful and naive feature re-computation for every object-like region in the image is well-known in the community, and there are several papers addressing the shortcoming in very similar ways (i.e. precomputing pool-5 features for the entire image once, instead of recomputing from scratch for each individual box). The Overfeat detecion paper [0] was among the first ones, for example, and it achieved similar speedups as discussed in the article, but the accuracy is not as great for various other reasons.

This article makes it sound like Spatial Pyramid Pooling is something new and amazing, but in fact this idea has been around in Computer Vision for a very long time (starting with this 2006 paper [1], which has now been cited almost 4000 times) and it has been successfully applied to pool descriptors over the image into fixed-size representations, ready to go into an SVM.

But this was the first time the basic concept was used as a layer in a ConvNet (by having variable-sized pooling bins) and it's nice to see the numbers.

[0] http://arxiv.org/abs/1312.6229

[1] http://web.engr.illinois.edu/~slazebni/publications/cvpr06b....

>This article makes it sound like Spatial Pyramid Pooling is something new and amazing

Hardly, the paper specifically states "Spatial pyramid pooling ... is one of the most successful methods in computer vision" and "SPP has long been a key component in the leading and competition-winning systems for classification".

I am still having trouble understanding the naive recomputation problem - aren't you forced to recompute due to the nature of convolutional filters? i.e. Thinking primarily in the first layer - an NxN frame is shifted right by one pixel column.

Doesn't this require recomputing the filter response? Or at least getting clever and using something akin to subtracting the "outgoing" pixel response and adding the new column's response for every filter?

Or is this only applicable in the larger context when you are basically convolving a whole network inside a larger image, after training?

My thoughts exactly - not to mention the fact that [1] was based on the earlier Spatial Pyramid work of Grauman & Darrell.

The language in the article is a bit zealous - Microsoft did well, but didn't "win" any challenge in Imagenet. VGG and Google seem to be the ones people are really watching / emulating.

While Google's this year's submission was good, it didn't seem to contain any particularly inspiring ideas, as it was mostly just a collection of small improvements on previous ideas. Personally, SPP-nets seem to be the most interesting idea to come out of this year's competition.

Reply on news.ycombinator.com