Strengthening our efforts against the spread of non-consensual intimate images

Being against harassment doesn't make you any less a believer in free speech.

Free speech is nothing to be absolute about. Things that were encouraged before can become absolutely abject later on - free speech in the public sphere must be regulated, and is in many jurisdiction (I'm French, I live in China - both are very similar in how they approach it in principle, China is more ideological when implementing but France can go very far when someone must stop talking).

I'm glad that as an absolutist, you managed to make a compromise, even if it's just for beheading. How can we convince you rape, insults, religious conflict, racism apology, or even treasonous discourse also can be regulated ? What is so good about stupidity that it has to be exposed so free to the public or what is so special about it that it can't be distinguished from intelligence ?

Banned from the internet? These scumbags need to be hunted down and be given long jail sentences, at the very least.

Yes, it's an awful and unacceptable situation. Different people have different ways of responding to harassment based on their risk acceptance and resilience: some people respond and fight back, some people hide, and some people seek help from others around them (the latter is, it seems to me, probably generally the healthiest approach, although I'm no expert).

What the parent post seems to be advocating for is that tech companies should automatically attempt to hide people from harm, and that they're best-placed to do that. I think it's doubtable whether they'll be able and willing to implement that in an effective or accurate manner (partly because it requires putting significant time, money and staffing into it, it can never be completely automated, and attackers -- especially in individual or small cases -- evolve as the representative mentions in the quote), and I don't think it respects or empowers victims, long-term.

It's also key to focus on the perpetrators - hold them accountable and figure out the reasons why this happened in the first place.

Free speech absolutists are probably going to wind up losing us our freedom of speech.

Its good that you're exploring what the limits are and that you have found a limit. Those need to be clarified.

You could upload part of the picture (with the nudity, or everything personally identifying, cropped out) and see if it matches part of any other picture online.

This frequently happens with GDPR data/delete requests. They will ask you to verify all the shadow data they collected about you is indeed correct before giving you a summary or promise to delete it.

The Meta-Zuck will verify: https://youtu.be/-XWedbTIcV4

It’s a real challenge - you can try and do more pre-processing on a submitter’s device to try and avoid mistakes or malicious use of the system. The big limitations are:

1. How much processing time can you put on what might be a mobile phone - it’s using MD5 for Videos, but StopNCII did also try other things that were just too slow for phones.

2. StopNCII chose to do verification on platforms after a match rather than do it up front to prioritize the privacy of the user. This decision came after a lot of feedback from victims and experts who represent them. There are all sorts of techniques you can use to process server side (thus saving the person’s phone’s CPU), but it deliberately doesn’t retain enough context to make them viable.

Or harmful ads for scam or viruses. Let's face it, Facebook did never actually care

It's very likely that the image can be reconstructed from perceptual hashes. Perceptual hashes make two promises, too:

* that the original image can't be inferred from the hash, and * that similar images should get similar (if not the same) hashes

and these are in serious conflict, with what's happened with gradient-based methods the last 10 years.

Yup, you got it, the content itself will remain only in the device, the hashing is done in-browser, and the only part of the original content that makes it into the system is the hashes. Once a platform that is part of the program downloads those hashes and is able to match content, you need to apply some amount of verification. It’s on the participating companies themselves to review the content that matches the hash, to see if it actually violates their policies on NCII.

Yup, PDQ (https://github.com/facebook/ThreatExchange/tree/main/pdq) and MD5- https://stopncii.org/faq/. MD5 is a cryptographic hash, which means that even a single bit changed gives you an entirely different hash. This usually precludes any attacks to try and deliberately generate collisions, but also means it's harder to match “benign” changes - for example, many platforms re-encode videos as part of the upload process (reducing resolution, changing formats, etc). However, many platforms skip re-encoding, so MD5 can have better results than you might think at first glance. The more bits you take from the original content, the harder you can make it to bypass, but the more you might have to worry about capturing enough that you might compromise the privacy of the original submitter. For the program, StopNCII picked a set of tradeoffs between those two tensions, and are keeping a close eye on how effective it is, and will iterate if need be.

Cropping does seem like a harder problem, in the sense that there's no "clever" (high compute efficiency) solution available. The only thing that comes to mind is the brute-force approach of comparing normalized small tiles instead of whole images, which would involve exponentially more compute effort. Interestingly, the article you cite mentions rotation and skewing, which make both clever and brute-force approaches even more expensive and/or less effective. Certainly seems like a target-rich environment for research.

True. I am not sure exactly what hash is being used (though others seem to indicate md5)

I have heard from colleagues in the past that consumers of illegal pornography used to (or possibly still do) commonly apply a negative filter or hue shift to images to avoid hash matches and flesh tone detection. This helps when combined with other stuff if someone scans their disk or they get raided and have too much stuff for law enforcement to look through properly. They then set up their screen or display settings to counteract the filter when viewing it.

Thankfully I have no experience with illegal pornography and definitely don't want to gain any, so hopefully I will never be able to confirm that firsthand.

The key difference is where the processing is happening. With Facebook, you can delete your account - with Apple's stuff, you'd have to get rid of potentially tens of thousands of dollars worth of equipment to escape the spying.

Consent during a sex act and consent to an image that you created continuing to exist after you knowingly release it into the wild are two different things. (Not including revenge porn or other nonconsensual recordings, obviously).

An actor who performs in an embarrassingly bad movie might wish they could refuse to let their work be used after the fact, but they already signed away their right to retract their work.

An OnlyFans model is also the creator of the content though, so they might have more leverage depending on OF's legal terms. If they still own the rights to all media they create and distribute through OF, then they could petition for removal based on DMCA or something (I'm not a lawyer but it sounds plausible).

Morally, I hope FB can help regretful adult models remove content of themselves if they don't want it to exist anymore. But practically, that will be very difficult to guarantee.

Consent? Yes. Images? No.

Any time? Can you withdraw consent after the fact and claim rape? If you are going to use categorical language, please be more careful about the language you use to describe a black and white world.

The platform can stop hosting some content immediately once consent is withdrawn, but it can't prevent that content from having already escaped.

It's probably not technically a "gray area" since the platform can't ensure the content doesn't escape. But certainly, content creators should be aware that withdrawing consent doesn't mean their content is suddenly secured.

During the act, yes. But I think the example here is basically: In your 20s you did some jobs as a porn star. 5 years later you want the videos scrubbed from the web. Is this within your rights?

That's what OnlyFans images are.

I wish there was a methodology whereby a person could remove their personal data from fb given that they were unaware of what they were consenting to and that it changed over time.

Ownership rights must factor in somewhere right? If I pay an artist for a painting, and they deliver it to me and take my payment I can then sell that painting to someone else, display it on the walls of my home, show it to my friends, or even give it away. If I pay someone to send me a digital photograph or even a video shouldn't I have those same options? The whole thing seems like a weird grey mine field of legal and personal rights.

I think that makes it an even more fascinating gray area, actually. It's exactly the important case where the hair needs to be split.

Using a technology like this for removing leaked private photos is one thing. Using it to enforce simple copyright is a whole other matter. If this were being deployed as a copyright enforcement tool, absolutely everyone here would be livid.

So, the question then is, if its used to censor a nudge image, but that image had been originally uploaded with consent, is that a matter of privacy, or copyright? And if it's merely a matter of copyright, is it appropriate to use this technology in that context?

I like the comparison of the program to hitting “report image” on a platform. Advantages of StopNCII is that you can report proactively, and to report it to all the participating platforms at the same time.

Misuse of the StopNCII platform is something that we (Meta/Facebook), UK Revenge Porn Helpline and the full StopNCII team discussed a lot. Once you have confirmed that a piece of content violates your platform’s policy, it’s easier to find other instances and filter out false-positives, because now you can use your platform-specific ML, or in-house photo detection algorithms.

I get what they're gesturing at, but I find a stronger emotional reaction against child porn than CSAM and it never once occurred to me that anyone thought this term implied it was in any way consensual or non-horrific.

No, a bunch of people needed a pretext to come up with some jargon so that they could feel special, so, among other things, they decided to adopt the unsupported and frankly ridiculous ideas that (a) the word "pornography" put the focus on the consumer in any certain way, (b) any such "focusing" was strong enough to be important or noticeable, and (c) anybody who wasn't desperately searching for something to complain about would ever care.

Another excuse they used was that the existing terms were somehow imprecise... and that they'd be able to come up with new terms that were (1) not equally imprecise and (2) somehow likely to defy the universal drift present in all human language, so that they would not end up with completely new connotations 5 minutes after first being used publicly, and completely new denotations shortly after that.

So they tried to rename things that had good enough names. Names that were searchable. Names that were meaningful both to people new to the issues and to people familiar with the issues. Not because it mattered, but because it made them feel important.

Agree to disagree, to me this seems like the euphemism treadmill paired with academic/professional lingo seeping into the public. Still, I'm not going to presume that someone knows more or has a more valid opinion on something because they use a fancy new acronym. And I severely doubt that anyone's mind will be swayed by an acronym either. Fortunately on both of these issues, I doubt many people need to be convinced.

George Carlin’s bit on euphemisms, which covers the evolution of PTSD, immediately sprung to mind when I read the headline.

There's also cold spray additive manufacturing, a 3d printing technique that involves spraying high velocity metal powder to build layers: https://en.wikipedia.org/wiki/Cold_spray_additive_manufactur...

Sometimes I think we go too far in an effort to be accurate and end up sacrificing impact.

A nude isn't automatically "intimate" either. Nor is nudity the only aspect that might make an image "intimate", or even the most important one. Nor does "intimacy" capture anything remotely like all of the reasons the material might be painful or embarassing for somebody. In fact, if I were looking for something vaguely silly to be offended by, I'd suggest that it was a bad idea to tell the subjects of those images what they should consider to be intimate or why they should feel offended by the images being shared... as the use of "intimate" tends to do.

Oh, and "images" unreasonably limits the scope of the media that might be problems.

Not all of the images are shared nonconsensually, either, at least not initially. And there are all kinds of forms and layers of consent. So, if you want to be all technically correct, you probably shouldn't bring that in either.

In other words, like all terminology, "NCII" is imprecise. In fact, I think it's a true step down from "revenge porn", even if you ignore the fact that you shouldn't change terminology for trivial reasons to begin with.

It is NOT an improvement. It's people arguing over minutiae so they can feel informed, feel like they're helping, and get a rush out of "correcting" everybody else.

I was in court when a defendant was brought up. He'd broken up with his girlfriend, but had her nudes. He knew another guy who had a serious crush on his ex-gf so he sent him a message and offered to sell him the nudes. Crushman called the police, bless him.

Prescriptivism rarely if ever creates useful linguistic evolution. The concepts have to change before you can stick new labels on them, or at least at the same time. And in this case what's being prescribed does not in fact tighten up any boundaries or "carve reality at the joints" any better than what it's trying to replace.