I mean, say you and I are at a neighborhood kid's birthday party with our own kids. Suppose further that you take some pictures, and then post them publicly. Maybe to your FB?
Well, I never really gave you permission to use my image on your FB. Let alone publicly. I know that I certainly would never consent to publishing images of my children publicly. Further, I suspect the birthday party's host wouldn't give you permission to post his/her family's images on FB either.
So here we are, with all of these people on your public FB. None having signed or even given verbal authorization for any sort of release. And now FB starts running FaceRec on all the information you just gave them. To top it all off, my family and I don't even use FB. Never have. So it really can't even be claimed that we consented to facial recognition via the terms and conditions of service.
I only outlined all of that to outline this, just because a person's image is in public doesn't mean that the person consented to the image being public. Especially images taken from private spaces, (like bday parties at some kid's house).