Using feature detectors and descriptors is only half of the solution. If you really want robust image recognition you need to use something like the vocabulary tree developed by Nister[1][2].

[1]http://www.wisdom.weizmann.ac.il/~bagon/CVspring07/files/sca... [2]http://www.cc.gatech.edu/~phlosoft/files/schindler07cvpr2.pd...