Cynically, I'm going to say you either find unsurprising matches (eg. photos of the same group of people taken on different days) or even more convincing and hard-to-detect false positives.

If your goal is explicitly to find something extremely rare - unexpected face matches in different places - you will be flooded with false positives even if the false positive rate is fairly low.