I’m sorry, at what point there was any confusion that Apple is going to use our phones to do scanning?
It was not universally understood that this would only apply to photos sent to iCloud.
It was not universally understood that this was only looking for previously known CSAM.
It was not universally understood that they were using some sort of hash matching so photos you took yourself would not trigger the system.
I understand if you consider the where more important than the others, but it is simply a fact that there was confusion on what exactly was happening here.
A lot of the contention wasn't about the specifics of their plan, but rather how subtle changes could vastly expand the scope of their plan.
"this would only apply to photos sent to iCloud." for now, until scope creeps.
"this was only looking for previously known CSAM." for now, until scope creeps.
"using some sort of hash matching so photos you took yourself would not trigger the system." well this one is immediately concerning even within claimed scope because there ARE going to be false positives that apple records some database. Millions of iphone users are going to have a non-zero "possible childporn" score.
They are building an engine for iphone users to self-incriminate. If they rigidly hold the scope to only what they announced and never expand, it could be argued that this is a reasonable concession to fight CSAM. However, in making the announcement, they boldly stepped past their existing hard line in privacy (local device content is private and not surveilled by apple), so it seems naive to expect that this announcement reflects the eventual scope of this self-incrimination engine for the next decade of apple updates.
I don't think this is right. The apple pdf on the features says
> With the initial match threshold chosen as described above, iCloud Photos servers learn nothing about any of the user's photos unless that user's iCloud Photos account exceeded the match threshold.
This implies to me that they are using some sort of encryption to prevent iCloud from learning even how many matches there are until the threshold is met.
The how helps show us how changing this system is not a subtle change. It isn't like they can flip a switch and suddenly they are identifying new suspected CSAM on people's phones. That would require a new system since the current one is only hash matching.
>However, in making the announcement, they boldly stepped past their existing hard line in privacy (local device content is private and not surveilled by apple), so it seems naive to expect that this announcement reflects the eventual scope of this self-incrimination engine for the next decade of apple updates.
This is an arbitrary line that is being drawn. These are photos that are marked for sending to iCloud. Whether the scanning happens on the phone before they are sent or in the cloud after they sent is largely immaterial when it comes to the impact of the code. People are acting as if the line Apple drew was motivated by technology. That was never the deciding factor. Technology is the easy part here. That line was only a policy line and that policy has not changed. Only photos that are sent to iCloud are scanned. If you fear Apple changing that policy going forward, you should have always feared Apple changing that policy.
This is a strawman. Identifying *novel* CSAM is a very hard problem to do accurately - the reason they can't flip the switch is because they don't have the technical capability. All of the other things people are concerned about are things that apple does have the capability to do.
EDIT to reply since at max thread depth: *novel* image detection was never on the table, I think you missed that word
> This is an arbitrary line that is being drawn.
It seems the vast majority of people in this thread disagree that this line is arbitrary.
EDIT: whether the threshold is "verging on impossible." depends entirely on the effective false positive rate. If apple's claimed 1 in 1 trillion rate is true, it's probably not a concern. However, I find it unlikely that perceptual hashes on portions of images won't have higher false positive rates when subject matter is similar (non-CSAM legal adult porn, or images of children in swimsuits, etc). If that rises to 1 in 1 million for these types of images, that's hundreds of thousands of people being falsely accused.
You just used this as an argument against this system. Why do you fear this if you don't think Apple can even accomplish this technologically?
>It seems the vast majority of people in this thread disagree that this line is arbitrary.
I would argue that people who believe that this decision crossed that line were being naïve to not have always known this was a possibility. I don't think this move brings us any closer to Apple scanning our devices for anti-government memes or whatever the fear is because what was stopping that was always more policy than technology.
Also to go back to your earlier comment, in the time since you posted it has now been revealed that this system needs to trigger 30 times before any action is taken. The odds of 30+ false positives is likely verging on impossible.
I'm a huge Apple fan and a Mac devotee of 30+ years. I love my iPhone. If this thing becomes real, I will go shopping for another phone.
This is absolute insanity. I can't believe they even thought about launching this.
I'm completely disgusted.
Maybe. I wonder how many people are choosing not to respond because they’ll only be voted down by the people who feel very strongly about this. I’ve curtailed and hedged many my contributions to this topic on HN because of this.
This is a common issue when social media is used to debate matters which are highly emotionally asymmetric.
* It was not universally understood that this would only apply to photos sent to iCloud.
Since the scanning doesn't happen on iCloud, this distinction is irrelevant.
"We are going to intrusively scan the subset of your photos that you care enough to back up to the cloud that we've been pushing to you for years" is pretty clear.
* It was not universally understood that this was only looking for previously known CSAM.
It was only looking for whatever is in an opaque database which, according to a third party we don't have any contract with, contains CSAM.
* It was not universally understood that they were using some sort of hash matching so photos you took yourself would not trigger the system.
Yeah right, I feel totally safe knowing that I won't be falsely reported to FBI by a "some sort of" hash matching.
Here's a hash function: f(x) = 0 for all x
It's "some sort of" hash, too.
It’s really not irrelevant. A third party photo library that avoids using the PhotoKit library would not be touched by the CSAM detector. There are many of these on the App Store.
One step further and store the photos encrypted, with a custom renderer that decrypts the content on the heap, and that would take some tremendous performance-hitting detection abilities it’s extremely unlikely to ever happen.
OK.
Misrepresenting each other’s arguments is not how you have a discussion.
I hope that's precise enough, and sure, everyone is welcome to make their own decisions regarding this.
- Whose fault is it that those points were not clearly communicated?
- Who wrote the perceptual hash matching code?
- Who is allowed to audit the code, the review system, and the hash database?
- Who updates this code?
- Who decides if your phone OS is updated?
- Who decides the iCloud upload defaults?
- Who decides if you are reported?
- Who asked for this feature?
To the extent that other parts of this story was explained to us by Apple, I did try to clarify some exaggeration in other thread.
This is ignorance in extreme.
> HN is a tech forum and I still saw plenty of people here worried about how they would get in trouble for having innocent photos of their own children on their phone.
They're confused about this. NeuralHash doesn't look for pictures of naked kids. It looks for pictures that are identical to the ones they've put in their signatures list.
The problem is that Apple claims that the signatures in their list are all pictures of sexually-abused kids, but we have no way of verifying that. Heck, they don't even have any way of verifying that. Everyone just has to take NCMEC's word for it.
The public does not know what the false positive rate is for 'average iphone user pictures'. As engineers we can be certain the false positive rate is not zero. This means that some number of iphone users are going to have non zero "possible child pornographer" scores in the apple database.
The false positive rate is crucial to understanding how concerning this should be. If the average iphone user has 1000 photos, and the false positive rate is the claimed 1 in 1 trillion, there is a 1 in a billion chance that you'll be flagged as a potential child pornographer. (~1 in the world will be falsely accused). This seems reasonable enough with the apple-internal screening step.
If the chunking and perceptual hashing functionally ends up having a much higher false positive rate for images which have similarities to the dataset (parents' pictures of kids playing shirtless, legal adult porn, etc), the false positive rate could actually be more like 1 in 1 million or worse. In which case there are potentially hundreds of thousands of people who will be falsely accused by this system.
How many matches will US judges require before they sign warrants for arrests, search and seizure of digital devices? If they are technically competent it shouldn't only be 1, but I don't trust all judges to understand probability well enough to require multiple matches.
Even that doesn't come without issue. How long before '1' becomes the value, because, say for example the number is ten, there's also a horrendous PR spin of "Apple has a high degree of suspicion that you have CSAM on your device, but since there's only 8 images, they won't do anything about it" - "Apple allows up to X non-reported CSAM images on Apple devices" is hard to represent in any positive fashion.
I yet to understand what happens to people who only have those synthetic positives? Regardless of what counter threshold is, can’t those people be hoovered up by a subpoena of counter >0 ?
That's really really really user-hostile design.