Russian photographer matches random people with social network photos
rbth.com
rbth.com
Rather than seeing this as an exposition of "people on social media are so much more bland than they appear" I found myself feeling "people on the subway are so much more beautiful and interesting in their 'real life'"
I feel like that might be backwards from how it gets presented elsewhere.
But, if you look for example at the blonde woman snarling a piece of cake https://birdinflight.com/ru/vdohnovenie/fotoproect/06042016-... (second last) I thought, that was charming and funny... But you'd never know that from under they grey fluorescent lights of the subway car.
It's really interesting to be confronted with the realization that every single person filling up space on the subway around you has a rich and lustrous story that brought them to that place and time.
Still, super creepy.
The dictionary of obscure sorrows[1] names this feeling "Sonder" The realization that each random passerby is living a life as vivid and complex as your own—populated with their own ambitions, friends, routines, worries and inherited craziness
1. it's a made-up word, not a formal English one. http://www.dictionaryofobscuresorrows.com/post/23536922667/s...
I'm sure if I was having a low day it could take it as a sort of "we're insignificant specks on a floating rock" but I was having a great day so tra-la-la :)
most are
Nothing beats the visceral explanation of a demo, though.
5 minutes later accidentally bump into her .. and start a conversation, armed with lots of private info about the person.
Or even more basic - thieves picking their targets - is this woman's purse likely to contain a lot of money and jewelry ?
I guess you could even write a tool which picks all the faces from a photo, and detects who's probably richest, most popular, most followed, etc.
But this is unavoidable, right ? We just have to accept it as a fact of life and adapt to it..
What do you have to hide? /s
http://www.nydailynews.com/news/world/man-disguised-woman-we...
http://www.mirror.co.uk/news/world-news/suicide-bomber-disgu...
http://www.telegraph.co.uk/news/uknews/terrorism-in-the-uk/1...
www.express.co.uk/news/uk/585432/White-man-burka-arrested-bomb-town-centre-Watford-army
I was working on a project with OpenCV a few years ago. After a few embarrassing incidents I had to add a kludge whereby if it detected a small face directly above a big face, it was to discard the big one.
It turns out that the torsos of portly women with large breasts make ideal face candidates.
Yes the looks are ridiculous, but most fashion starts out that way. People often look for ways to stand out and being "retro" could enhance the cool factor.
What does someone care if a mugger's facial recognition software "flags" them? And who exactly is being flagged? And if a security surveillance system flags you what is the consequence? Who does that consequence apply to?
http://www.bbc.com/news/technology-35111363
> "What you can do now is link your face-recognition system to Facewatch and it will pull down the watch list that's relevant for your premises and your group," he says. "Then, if someone walks into your premises, it will send you an alert."
So if you have a criminal record, or your face somehow ends up in that database, you could find yourself followed around shops and asked (or told) to leave. These sorts of massive watch lists aren't feasible with merely human recognizers.
Think about it in reverse. Think about every photo in existence anywhere that has you in it, or has your stuff in it. Imagine all of that being tagged and cataloged as being tied to you. A selfie your friend took in your house with your kitchen in the background. Somewhere somehow the data is crunched, algorithms are leveraged, and these things are figured out. A lot of this isn't even a particularly hard problem, your social graph is already known, your address is known, people share their locations all the time, the people you hang out with is known, it's a simple problem of deduction to figure out all this info. And then there's an amazing degree of data that can be extracted. Imagine a database containing a list of everything you are known to have in your house, and when, and changes to that. Imagine tracking your whereabouts and who knows what else by the presence, or absence, of your self or your accouterments or your car or what-have-you in various locations in the backgrounds of other images.
We really don't have a good sense for the implications of this level of data becoming available to anyone and everyone.
https://www.youtube.com/watch?v=IqeOnfQmZPY
He stalks people on social websites and then strikes conversations with them.
This novel is set in a world that takes current social and sharing trends and extrapolates out a dozen years or so. Depending on your thinking, it could be seen as an extreme or a reasonable guess of what might happen in the future.
In the world of the video game, people did adapt to it. However, there was a faction that actively tried to destroy that system.
http://www.huffingtonpost.com/2011/06/01/facial-recognition-...
For a woman with a lot of diamonds in her purse i would suggest wearing a burqa, it would prevent most biometrics from being spotted by the cam. On the more serious note, what methods did he really use?
I don't think so, if you're happy to remain off social networks.
Personally, I see nothing wrong with one internet picture for all these social sites. I wear huge women's sunglasses on my one picture I put out there. It's probally against TOS, but who really cares? There's always another website.
That said, I've never had anyone pick me on my looks.
I need to put in time writing, or talking. It's a pain, but I've never liked knowing anyone in the world can download my image.
Actually, I don't like anyone taking my picture. My only pictures in school was k through 8th grade. I wasen't present for picture day in high school. College did get my mug.
My hope is it will become socially frowned upon to take anyone's picture without their consent.
Whenever I see that Shutter bug at social functions--I just with they would put down the camera, or take it out at end of the hootenanny?
careful with that edge, you might cut yourself
I'm not so sure. I think its very possible that the person in the social network photographs is actually far closer to the person they really are than the photo of them riding alone on the subway.
(I really like this sort of "subversive art", that has deeply important educational aspects showing people how the world really is in interesting ways...)
Anyway I don't know anybody who busts out their truest expression of self to ride the subway.
Sometimes I suspect my early morning public transport "if you don't turn that stupid ringtone down I'm gonna stab you in the face" self is closer to my "truest expression" than my "look at me having fun being the life of this party" social media fake-self...
So I think it cuts both ways.
I seem to be alone in this, but... they don't even look like the same person to me. Do we have any confirmation that the pairs show the same person? About 10 million Russian men look like the guys in the second pair of photos.
Most of them are clearly the same person.
That's how I see it too, but that may be tempered by my personality. When I'm on public transit or in a crowd, I do my best not to stand out (which is difficult being 6'4" and 240lbs) but when I'm around people I know, I will smile and have fun. When I go to a concert alone, I'm also a bit closed off and tend to dress blandly, but if I go with a friend or a group I'm more lively and interactive, and dressed to suit the music. It's all about being comfortable with those around you, at least for me.
So yes, on the subway I'm the dull guy in the jeans and t-shirt, and on social media I'm almost always with friends in a photo so I'm smiling and having fun.
Nothing out of the ordinary in these photos if you ask me.
Maybe it's that the picture on the web was chosen. Someone put thought into that, they said "oh hey I look good here", it's kinda like an ideal, or at least an above average view of themselves.
It's more surprising when people "overshare" and detail intimate information about themselves IME.
Who looks through photos to choose to preserve and doesn't pick the one's in which they look "better", doubly so when sharing those photos in public?
Something about the surprise of it? It feels like a surprise, it feels like something that shouldn't be so easy.
You can't generalize from the particular to the whole because large differences in quantity add up to qualitative differences, and machine processing makes laborious correlations and connections easy to tease out.
There's similar problems with automated number plate recognition, public CCTV, non-cash transactions, etc. They all add up to massive potential privacy violations despite being fairly innocent in isolation.
Give someone your phone number, and they can't do much more than phone you. Give them a phone book, and they can phone someone given a name and address. Give them a phone book's worth of data in CSV, and they can deanonimize phone call records. Give them a list of phone numbers that have e.g. called a suicide helpline, and now they know the names and addresses of people who are likely to kill themselves. Without machine processing (i.e. with a book rather than CSV), this would be difficult; without people sharing the relatively innocuous detail of their phone number, it wouldn't be possible (leaving social engineering aside).
So if we want to live with computing, I think we need to learn to live with capabilities of computing and the nature of the casual structure of the physical universe.
Forever, if someone recognized you, chances are you recognized them. People were either within some extended social circle or they were not, and this social circle was face-to-face. Now, we have something of an arms race going on with respect to anonymity, or the lack thereof. Maybe someone will scan me and identify me. What's my recourse? Either I go about in disguise, or I scan everyone around me in order to equalize things.
How could all this not feel strange or wrong? Based on the norms of 10,000 years of human society, it's either a-social or downright anti-social.
I think for most of human history, humans knew the majority of the other humans within travelling distance and people were wary of strangers because there was no reciprocal trust relationship.
Given enough information, everyone could approach cold reader's peak performance at a glance with additional biography. Sort of creepy - mostly because you can use such information to manipulate people.
But like any knowledge it can be used to break ice, strike interesting conversations and broaden one's circle of acquaintances. Or it could be used to build new tribalism.
The most problematic part is being able to cut loose and erase the past. This used to be much easier in the past - you just moved far enough and most everybody didn't know what you did.
Why are you surprised that this data can then be used by a third, or even a fourth party?
It isn't what they think they do, but it very much is what they do.
Now tell me how uploading things to such a database is not a willed action.
I agree with you that there is the possibility of someone else taking such images of you and everyone around you, through surveillance cameras or whatever, but for something of the scale shown in this article you'd still need quite an effort to create such a database without the help from the people themselves, especially if you want to link it to other personal information.
Because people don't own their own clouds, or have privatized social networks, they are forced to upload things into databases that others control. Who controls the privacy defaults typically have the most control over where the data flows thereafter. And it's all out of the user's hands; they weren't given a choice, not a real one, because the incentives for the system lead you to accept the lack of control, or spend your life neurotically trying to keep a lid on everything, or live as a technological recluse. There's no other way to live today.
People are freely giving away the images for the database linking it all to further personal data. If they did not do this, creating such a database would still be a massive surveillance effort, albeit probably still possible if someone really wanted to do it and had the sufficient resources.
I say it's still possible to stay anonymous if you want to.
When I spot such a camera I stand to the side so they have to ask me to move in front of the camera. Which is when I politely refuse to participate. They don't like that very much. Last time, the person made a "well you might not be able to refuse next time" remark. He may be correct, or not, we'll see.
These are cases where the images should never be uploaded to a public database or shared with any other parties. However, given the increasing use of SaaS and cloud providers, exposure to some third-parties is highly likely. Any consumer information hitting the cloud is at high risk of abuse.
In addition to those scenarios you have cameras behind ATMs, cameras embedded within grocery store self-checkout lines, etc and so forth. It has already become difficult to control your image and, in some cases, prevent that image from being combined with other personally identifiable information.
Even gym memberships require this in some places. Soon one would need to provide facial 3D scan just to buy groceries.
And in most places it still wouldn't be socially acceptable even if it may be legal.
And I'm trying to figure out why this apparently feels so wrong and creepy to so many commenters here. I don't feel any of that. As far as I'm concerned, it feels... totally normal. Maybe I'm wired wrong or something, but I really don't see anything weird or creepy about learning from publicly available datasets (AKA observing your surroundings?). Nor do I see anything wrong in leveraging computational infrastructure to learn more from those datasets.
The way I see it - a hypothetical "creep" who takes a photo of someone and matches it with their public social media profile is fine. Where I draw the line is them attempting to put that into action through stalking, harassment, etc. because it hurts the other party. Trying to break into data sources that are not meant for public is of course a violation of personal privacy, which is wrong. Also, I consider a person with obsessive interest in another to have a problem that probably requires medical attention.
Are people so afraid today that the fact that another person notices them makes them uneasy?
https://commons.wikimedia.org/wiki/Commons:Country_specific_...
Additionally he quite likely doesn't have permission to post the profile pictures due to copyright.
Also, the neural net may return a certain percent of false positives and there may be no particular way for this guy to tell. The impact is small here and would be larger in other circumstances.
I have a significant number of friends who look at me like I'm speaking in Klingon when I tell them I'd rather not be tagged in their Facebook photos.
Occasionally a family member wants to post something but there's seldom a problem.
Most of my friends don't post anyone's photo on Facebook even though they use it a lot.
It is not voluntary.
Some people will not have a public profile picture but they too could be subjected to it.
Looks as though it's only available in Russian and only works for VKontakte (the social network).
Warning: It highjacks your back button.
On one side it is creepy, on the other it would be nice to have your digital assistant suitably remind you the name of the person who is walking towards you and recognizes you but you cannot remember who they are. A quick summary of who they are and perhaps a summary of your last interaction whispered in your ear could be handy. Or even a couple of relevant recent headlines about their company, etc.
Of course creepy side is pointing at random stranger on street and instantly having their digital dossier overlaid on their face.
Personally, I believe that your tools are extensions of your body.
Tools are an extension of your body, yes, but you don't go using your hands to feel up someone because you needed more texture data. You don't stare at someone for a long time because that's rude and intrusive. Same goes for using technology like this.
Do you feel uncomfortable about the fact that most of your friends don't even know your birthday date or phone number, and have both noted down somewhere (be it a phone or indirectly via profile info of your Facebook account)? Is it weird for you that some may even note down things you like so that they can check them out when looking for a good gift for you? Do you feel "like a product or mark" when a conference badge you wear displays your occupation or interests (sometimes used as an icebreaker)?
(Also, you shifted from just scanning to using the results to "try and strike up a conversation"; those are two different things.)
> Tools are an extension of your body, yes, but you don't go using your hands to feel up someone because you needed more texture data.
By scanning I'm not touching you. That would be a violation, yes. But why should I let you be a master of my eyes, and the (hypothetical, at the moment) tools I use to augment them?
People will get used to it, like they do with everything we've invented since dawn of time. Social conventions will adjust. Just because my eye implant can tell me what's your name doesn't mean I get to stop you on the street and start talking to you, no more than it is an acceptable behaviour today.
And honestly, scanning from a distance is still incredible invasive.
You can be the "master of your eyes" but don't get offended if I or others think you're a creep. You may not be touching me but you're definitely noticeably staring. The rejection Google Glass actually gives me some faith in the decency of people, but you're probably right, society at large will adjust because the profit motive and law enforcement capabilities for this sort of technology are too great for them not to become entrenched by media and government interests. Personally I find it inhumane and repulsive, but I'm old fashioned like that. I prefer more organic human interactions, nuance, getting to actually know someone beyond a readout on a screen. I'm a private person in an increasingly public circus of society.
These photos are matched too well, and I suspect he did it manually. Some of the train photos are so bad, there's no way they got automatically recognized.
Not to mention the resources you would need just to get the data on 280 million users, let alone train a neural net on it, and make it work with a 70% success rate.
I think it's a viral way to get app installs. The app reviews are really bad btw, it doesn't work.
Incidentally, is there anyone who could ELI5 how picture data like this is treated in order to run these algorithms on it?
> In 70 percent of cases, he was quite easily able to identify the people he had photographed.
VKontakte has 280m+ accounts and is the second most popular website in Russia (https://en.wikipedia.org/wiki/VK_%28social_networking%29); the Russian population is ~144m. I can't find any decent estimate of how many Russians use VK, but it seems to be at least as dominate in Russia as Facebook in the USA, which is to say, ~71% (http://www.pewinternet.org/fact-sheets/social-networking-fac...). So from non-VK usage alone, he should be able to identify no more than 70-80 percent since you can't identify those who aren't on VK in the first place.
Facial recognition is a notoriously difficult machine learning task, and doing facial recognition on the VK population of St Petersburg (>249688 * 0.7 = >174781) will be challenging even with extremely good facial recognition. I don't know what this 'FindFace' application is, but I'll be charitable and assume it matches the 2015 state of the art, "Facenet: A unified embedding for face recognition and clustering" http://arxiv.org/abs/1503.03832 AFAIK, with a best-case binary classification accuracy of 99.63% (https://en.wikipedia.org/wiki/Accuracy_and_precision#In_bina...):
0.9963 = true positives + true negatives / true positives + true negatives + false negatives + true negatives
If each person = 1 profile photo, then to classify a photo of someone there would be 1 true positive, 174781-1 true negatives, and unknown number of false negatives.
0.9963 = (1 + (174781-1)) / 1 + (174781-1) + x + (174781-1) 0.9963 = 174781 / 349561 + x 0.9963349561(1/174781) = x 1.9925943 = x
There would be ~2 false negatives for every true positive. So there would be a <1/3 chance the photographed person is not on VK at all, and when one searched for them when they are in fact on VK, they would only show up <1/3 of the time (the other two-thirds their subway photograph would not 'match' when tested in the dataset and it would be a false negative, and it gets much worse if the face detection has one of the lower state of the art accuracies on the other datasets). So on net you would expect to turn up a person <07 * 0.3 = <0.21 of the time. (It gets worse if we take into account the false positives as well but that particular paper doesn't report precision.)
There is no mention of any attempt to validate the findings, so I think for many of his results, he may have merely found a similar-looking person (possibly in some cases, look-alikes http://www.nytimes.com/2014/08/26/science/looking-at-twin-pe... https://en.wikipedia.org/wiki/Look-alike ). Look-alikes are far from identical twins, so it's not necessarily surprising that
> Often, there was a striking difference between a person’s real look and the image they projected on their social network profile: a shy and grim young man might appear to be the life and soul of the party and a lover of extreme sports on his VKontakte page.
This could be due to care in social presentation. Or it could just be that they're not the same person in the first place. (The photos are not that persuasive. I'm sure there's more than one grim young men with widow's peaks in St Petersburg, or young women with curly black hair.)
But really, you're being way too generous. 99.63% accuracy was reported on a relatively constrained dataset[1] when being asked a yes-no question: "are these two faces the same"? In our case, however, we're speaking about a search algorithm that tries to match something in a huge database. The accuracy will be way, way worse.
But, the point being missed is that OF COURSE people don't look the same "in day to day life" because those social media pics are just a moment of one day. If the prof happened to meet the person moments after the social media picture was taken then ... he would have concluded that "people present themselves truly as they are in reality" ...
My point, the point ignored in the article, is that Social Media is a self-reflective, self-curated snapshot. Not a continuous, objective representation.
Or that the guy about to get off the subway earlier thanked his friend for giving him that money he owed him.
And they won't even need retinal scanners. Facial recognition will probably be good enough.
Also consider that your local grocery store clerk probably knows your eating and drinking habits better than you know them yourself.
You could easily reduce the search space to match a face taken by a CCTV camera by using cell phone location data to limit it to people in the general area...
The difference from before I guess, is that it's less in your face, and can thereby happen without people noticing as easily.
Not for everybody all the time, though...
I'm curious naskwo, it says in the tour "We don’t synchronise any content to any external sites or social networks." - this implies you don't have off-site backups?
Second, for the birthdays calendar do you do anything like attempting to recognise photos of birthdays and log them, or use image meta-data to do the same?