Facebook’s image outage reveals how the company’s AI tags your photos | Hacker News Reader

Facebook’s image outage reveals how the company’s AI tags your photos

theverge.com

183 pointssaravana8557 comments

Lots of comments about how this isn’t new, or how it’s related to accessibility. The article mentions all this. That’s not what the story is. It’s not a very deep story, but I don’t think it’s quite so trivial as is being suggested.

The story is about this info being exposed to users just browsing the site, and the reaction that elicits. Someone seeing the tags inline in their feed by a glitch will tend to have a very different response than someone who is asked to view source and look for it.

I believe this is relevant because a regular conversation around here centers around the idea that many folks in tech circles find it creepy or unhealthy for society to have so much info being collected and processed for profit by giant corporations, and how most people outside of tech circles seem not to care at all.

I agree with your point entirely but I wonder if more people outside of tech would care if only they understood what these companies were doing with their personal data.

Yes I believe the number of users on facebook would drop incredibly quickly

That is an incredibly charitable interpretation of the word "reveal" in the headline. Every piece of information can be "revealed" to someone by that definition, it's happening all the time, and it wouldn't be a newsworthy event.

"Reveal" means to show something that was previously hidden from view - what has happened here is pretty much the textbook definition of the word.

The implication is that a secret is being revealed, possibly a dark secret to worry about. But FB and others put content in alt tags for people to read when they can’t see the images (for example die to blindness), it is much less dramatic to “reveal” something that was intentionally put there for people to read.

is image alt text really that hidden, even if facebook doesnt show it on mouse hover? any screen reader would have used the descriptions, no?

Yeah no. It's not like people have not seen tag suggestions before, or don't realize the relevance of recommendations etc etc. Even the most tech-illiterate facebook users i know all know that.

This is just the alt-text associated with the image and is there for accessibility purposes. That text is always part of the page even without the "glitch". This to me is a good use of AI as it provides a richer experience for those who can't otherwise view the images.

I suppose it might make some people aware, who weren't otherwise, that their images are being processed for content.

Much more interesting and readily available though are the "interests" Facebook assigns to you, which everyone can access: Settings | Ads | Your Interests. These hint at much deeper analysis that is being done with data, and data from other sources beyond just analyzing pictures for content.

This is the golden goose from Facebook's perspective. Their whole business is based around having a giant ontology of subjects and slotting you into this giant graph, then selling access into that ontology.

That has been the basis of targeted advertising since well-before the advent of the internet, but now the ontology is much, much richer and the people are providing the data used for the AI-driven slotting all on their own.

I was unaware of that section of Facebook's settings, so thank you for the tip!

Looking through it, I am far less concerned with what Facebook knows about me, but I am not a daily user of the platform, and so it is probably understandable why their "Interests" as flagged are so generic (and also some are just wrong).

Maybe I'm used to my slow internet connection, but I've been seeing these tags before the images load for a couple of years now - I thought it was common knowledge.

Same here!

Things like that often come as a shock to normal users when they actually see or realize the amount of knowledge these companies have. The big silicon valley companies are literally peeping toms that the society yet tolerates.

This always surprises me. How can you live today not understanding/predicting the vast scale of corporate/social media analytics...?

If major hedge funds are monitoring parking lots with satellite cameras you can bet damn well facebook is using image recognition in its targeting

Besides - both Apple and Google openly reveals that they auto-tag images by allowing you to search your albums on these sites with the tags as a feature.

And Microsoft too

No one I talked to was surprised. I've always thought it was pretty neat. As a human I tag people in my photos, I let the machine do the boring stuff (this photo contains trees)

Slow internet connection or 150+ tabs open in browser? :)

This stuff is in the HTML, you can see it with web inspector, or I think just by hovering your mouse over the image because that looks like it's in the title attribute for the image. This isn't it like something new was exposed inadvertently. I'm surprised The Verge wrote an article about this.

I think it's less about "look at these [creepy] things" and more about "hey guys, you saw all this text when Facebook broke, this is what it is." At least, that's how a lot of my non-tech friends seem to be taking it.

It's creepy on Facebook, but with any cloud photo storage, like Amazon's or Microsoft's, being able to search for "ocean" or "plants" in your own photos is really fucking cool.

Note: You can do this in Apple's Photos.app and the analysis is done 100% locally on your computer.

This is what i see, when i inspect a element in ff.

.d8888b. 888 888 d88P Y88b 888 888 Y88b. 888 888 This is a browser feature intended for "Y888b. 888888 .d88b. 88888b. 888 developers. If someone told you to copy-paste "Y88b. 888 d88""88b 888 "88b 888 something here to enable a Facebook feature "888 888 888 888 888 888 Y8P or "hack" someone's account, it is a Y88b d88P Y88b. Y88..88P 888 d88P scam and will give them access to your "Y8888P" "Y888 "Y88P" 88888P" 888 Facebook account. 888 888 888

See https://www.facebook.com/selfxss for more information.

It’s in the alt attribute, like it should.

You could also see this if you viewed source when it was fully working, of course. And occasionally before images loaded in if you were on a slow connection. I don't think it's any great secret, but still interesting to see.

I was thinking the same thing. I've noticed this multiple times in the past, but I guess alt-tags are too archaic for Average Joe.

It happens sometimes in Instagram without any big outage

I mean, journalists could also see this if they employed people with disabilities that required them to use such accessibility features. Considering it took an outage to show them I'm guessing few to zero companies do.

Not only is this nothing new as other commenters have observed, these tags have been used by screen readers and other accessibility services for awhile. While we can talk all we want about Facebook using their all seeing eyes for evil (which I agree they overstep a lot of boundaries), these tags are very useful for people who may require them and should not be scrutinized as an invasion of privacy.

What I think is more interesting about this outage is that Instagram and WhatsApp were affected. This means your images are stored on the same servers, and possibly subject to the same AI scanning. I wonder if WhatsApp images are E2E encrypted...

IMHO by now if you're still on Facebook you've made a conscious decision to let a business with an awful track-record of respecting its users' privacy basically have carte blanche over whatever you give to them.

I'm sure there are a few folks out there who don't realize this, but most people I talk to about this say they a) hate it but b) don't want to leave the platform because it lets them keep in touch with people. To them, it's worth it. Not to me.

I know for sure that this technology improves life of visual impaired persons as special software will read respective text.

It’s funny how what purely is an accessibility feature done to help “differently advantaged folks” (how these journalists would say) is being turned against them by the media.

Reminds me how they are vilifying open APIs and explaining them as a way Facebook “gives away your data”.

Reading some of the comments right here, it’s interesting watching some eat it right up and go right along with it.

Facebook is doing a lot with user data I have serious problems with. An accessibility feature like his probably ranks dead last or close to it.

it's not about what it means for this community, where we all know the shortcut to the dev console in chrome. this is about regular users of Facebook having their own "they live" moments on social media. where they can see plainly how Facebook regards them as consumers. "oh just show him pictures of DRINK and TEXT, that's all he's into anyways".

"They Live" is a great comparison. Of course this isn't new to someone on HN who follows this technology and the companies using it for a living, but for my uncle who just posts pictures of his dog it's a bit shocking to see that the computer knows that it's (likely) a picture of a dog next to a tree, even if he was aware on some level that computers these days are capable of that kind of classification.

Just alt-text. Granted it is a slight insight though, as I assume it's added automatically, but I expect they pull a lot more info from people's images than just what the alt-text shows.

There's also an extension which overlay the alt tags over images: https://github.com/ageitgey/show-facebook-computer-vision-ta...

>> Everyone knows the bit in The Matrix when Neo achieves digital messiah status and suddenly sees “reality” for what it really is: lines of trailing green code.

Given that the Matrix was a simulation, what it "really is" can't have been trailing green code. What it "really is" must have been electric charges stored in memory circuits. What Neo saw was his own mind's interpretation of the software and hardware underlying the Matrix.

Though of course the point about the green text is that it was a spiffy effect, and not a supposedly accurate representation of what computer memory "looks like".

I believe the green text is also taken from a recipe book but the designer won't reveal which one.

Looks like alt tags for accessibility purposes. Seems like a win win for all users. Easier to find photos and the ability for visually impaired users to get context on what their friends are sharing

This is nothing new, I've noticed that several months ago and talked about that with one of my security friends and he confirmed that Facebook is tagging pictures at least since 2016.

Here's the announcement, even: https://code.fb.com/ios/under-the-hood-building-accessibilit...

I think I've never seen worst GDPR compliance : big "I Consent" button with no negative counterpart, except a link to the cookie policy where the "opt out" section invites you to disable cookies in your browser and manually edit local storage. Is that a joke ?

If you're using mobile Facebook like - 0.facebook.com then this is well known for you.

And in other news people using screenreaders have been viewing these texts the whole time...

"Reveals". God journalists are lazy these days

So ... feed images into Facebook to break Captcha?

Dumb article. It's the alt text.

This is true. Maybe this was posted to remind us of how unknowledgable the average computer user is regarding the web?

Why do they put this data in the alt-tag in human readable format? For what purpose or use? SEO on a personalized platform with very limited access by Googlebot? Just to show they can? It makes no sense to me.

For accessibility?

I'd be surprised if they even care about that, but who knows, it's a possibility

It's a legal requirement in some parts of the world. Oh, and it means a bigger potential userbase for them. Plus it's just generally an all-round considerate thing to do.

Why would you be surprised? They put in effort for their iOS apps to work properly (and the alt-text tags are used for Instagram in "blind mode" for example).

If you're a company that wants personal data from everyone, you want personal data from everyone.

You'd be surprised if the largest content delivery platform in the world cares about being accessible to the broadest range of users?

https://code.fb.com/ios/under-the-hood-building-accessibilit...

The purpose of the alt tag is to provide a sufficient description of the image if it cannot be displayed, like it happened here. Everything went fine.

Reply on news.ycombinator.com