Something to note here is that in the hash collision that was discovered, the two images look nothing alike. One is a picture of a dog, the other is blobby grey static.
I know that that "think of the children" is a meme, but I think this illustrates the point for apple. If you have a croped or modified image of CSA the system will identify it. As long as your image is different enough from CSA, you are safe.
The point here is that Apple is specifically looking for matches against known CSA material.
If someone can demonstrate that a legal NSFW image (eg. regular old-fashioned pornography), can be collided with a legal, completely 100% SFW image then I'll be concerned.
But until then, this looks like a reasonable and supportable way for finding CSAM in real time.
Look at this other collision: https://twitter.com/SarahJamieLewis/status/14282060881181491... An attacker can send you an innocent looking picture that embbed some CSA material and you get swatted the next day.
Looking at those two images its plain to see why they are identical.
And if someone is sending you CSA material, isn't that the point of this process. Apple identifies it as CSA, can give you a warning its sensitive, and identify they authorities that people are sending CSA.
Again - this seems like a win. If Apple can automatically identify CSA material, en-masse thats good.
edit: It looks like they are different URLs, but Twitter only allows people to see replies if they are logged in, so I can't see that example.
edit 2: On further thought, if someone can use CSA material and produce an innocuous image with a similar hash, if they send that image to you, its still proof that the sender had CSA material. Again, its still good.
Consider if the honeypot images (manipulated to match CSAM hashes) are terrorist recruitment material for example.
Bonus points if you match poses, coloration, background, etc.
I think that's a perfectly acceptable outcome, since anyone with the hubris to both possess CSAM content and create replicas of it especially deserves to be arrested. Do you see a more problematic outcome here?
What if the visual check gets accidentally signed off on, or even gets fraudulently marked as positive by a burned out/lazy/competent-but-distracted employee? There are just so many failure modes for this dragnet that all end with innocent people suffering a very difficult process, even if they don't eventually land in prison. I don't think it's unreasonable to not want to be volunteered to participate.
And in retrospect, you don't need to match that - you just need it to appear obviously pornographic after the blur is applied in order to get past Apple's reviewers. After that, the lucky individual's life is in the hands of the police/prosecutors. (I have to imagine that both real and faked cases will look pretty much like "Your honor/members of the jury, this person's device contained numerous photos matching known CSAM. No, we won't be showing you the pictures. No, the defence can't see them either." Can you imagine a "tough on crime" prosecutor taking the faked case to trial too? Would the police and prosecutors even know it was faked?)
https://twitter.com/SarahJamieLewis/status/14280837442802565...
Unless you're looking to build a porn dataset and you're don't care about copyright. Porn is industry where exabytes of material are produced and published on internet almost every week.
Who will agency come to? To some OnlyFans creators?
Is this just as unacceptable as the CSAM scanning? Should all public photography be banned, in order to reduce the risk of false positive identifications of innocent people as criminals to zero? Or is that an acceptable degree of privacy impingement for the good of society?
Is Apple’s implementation an acceptable trade of impingement and risk, for good for society? We do live in a society, and so zero impingement upon privacies is never going to be acceptable (sorry, free/tech/libs). But instead of discussing whether Apple’s approach violates privacy minimally or not in order to get the job done, these discussions here just keep circling the drain of “putting my privacy at risk by any degree is never acceptable”, when that drain is cemented shut by the existence of society and will never lead to a valid outcome.
Apple's implementation is more like a stranger sneaking into your house if they suspect you're up to something bad. That would be equally unacceptable - we have warrants for a reason.