I think only some parts of a CAPTCHA are challenging you (data is labeled sufficiently for mistakes to be considered a failed solve), and the other parts are still in the process of being labeled, which you are helping to accomplish, so those have no influence on the immediate outcome.
At least that's how the old "type these two words" CAPTCHAs worked. It was crowdsourced human OCR of whatever text the machine OCR couldn't make sense of. I'm not sure if "find the bus/motorcycle/crosswalk/light" is the same way, but perhaps, and it does seem to offer leeway when there's only a few pixels of that item in the frame.