samrolken

It seems like this system was designed specifically so that this would be impossible, and such a feature would go against what seem to be design goals of this system.

They went through the trouble of making this whole “private set” matching so that the client does the matching but doesn’t know the result of the matching. Only the server can (once enough matches are made that the key is available).

> once this system is in place it will be used for anything, not just photos

The system seems designed to make it hard to use it for anything else. How will a hash driven by a visual perception based neural net be used on ZIP files? How can you add ZIP files to your iCloud Photo Library?

Sure, Apple could possibly do any number of bad and worse things. It’s a matter of trust that every time we update our iPhones that the update doesn’t include a ZIP file scanner or a blasphemy-scanner. This has always been the case, even before the introduction of the CSAM voucher mechanism.

Even if we accept that your image of your kids in the bath will match the hash of a known and identified CSAM picture (a real stretch), under this system the voucher payload does not contain the private key to decrypt the picture, so nobody will be able to evaluate a photo or flag it as anything. Humans have access to a “visual derivative” based on the perceptual mechanism used to create the hash, but not to the actual photo.

Other companies have CSAM scanning and reporting with much fewer safeguards and the “my kid in the bathtub” scenario hasn’t seemed to actually be a problem.

Fortunately that’s not how it works. You would have to have a whole collection of known and identified CSAM to be classified as anything at all with the system Apple has announced.

You’re missing the threshold that is part of this system. You would need multiple hash collisions across multiple photos to trigger these mechanisms.

A false positive will not have any effect. The threshold system they have means that they won’t be able to decrypt the results unless there are many separate matches.

Source?

It is a hash created with ML. So it’s both. But yes, it only matches already known material.

An image being erroneously flagged will not have any effect. Until the threshold is reached, Apple is not able to know if or how many images have matched.

And even then, the voucher doesn’t include the key to decrypt the photo itself.

Apple has documentation that explains all of this.

> If the system detects any matches

This part isn’t true. Unless a threshold of multiple matches is reached, Apple won’t have a complete key to decrypt anything.

Apple has documentation explaining how it all works with a database of known images.

Apple’s documentation discusses how they use a neural network to generate the bits which get turned into the hash.

The detection of sexual images in kids’ messages doesn’t use the same hashing setup as the iCloud Photos detection feature.

If you think Apple is going to install something onto your phone to scan everything for wrongthink and send it to the CCP, you shouldn’t be installing their updates or using an iPhone anyway. They can already do that at any time. This feature doesn’t make it any easier. On the contrary, this feature seems specialized for its purpose, and to slide down your slippery slope they would need some different and more general kind of spyware.

You can opt out by not using iCloud for your photos. If you prefer, you can install Google Photos or OneDrive onto your iPhone and let your images be scanned in the clear in the cloud instead.

Your kid in the bath won’t be in the database being matched against.

At any time Apple could turn evil and include invasive spyware that sells you out. If you’re afraid this can happen, this new feature doesn’t make it any easier. You shouldn’t use Apple products at all if you distrust them to that degree.

The neural net is specifically designed to handle such cases. This is covered in Apple’s documentation about this.

It seems like it isn’t possible to store images in someone’s iCloud Photo Library without their involvement. This would be a major security incident and immediately patched, if someone were able to find a way to do this.

Apple never gets to see your photo. Even if the photo matches the database and even if you have enough photos matched to reach the threshold that lets Apple decrypt the hashes, the photo itself is not decrypted or sent to anyone as part of this process. Only the related metadata.

Microsoft and Google hash your unencrypted photos on their own servers. Apple could easily do the same… I wonder why they didn’t…

What does that mean?

It remains to be seen if this is possible at all… let alone “easily” done.

Have any other big tech companies that scan for illegal images been shown to be vulnerable to this?

An attacker who has a way to place multiple illegal images onto someone’s phone and into their iCloud Photo Library without their consent… it’s an interesting idea but how is it possible?

This is only applied to photos being uploaded to iCloud.

It sounds like this is a feature of the encryption used. Apple will not have enough of the key to decrypt anything unless there are enough vouchers which match.

If you are already planning to charge people that are signing up for free, then it is relevant at this point in time.

How do you plan to handle:

Today: customer signs up for free.

In six months: you start charging fees

In six months plus one day: the customer sells something, incurring a fee even though they signed up for free.

The pricing page mentioning “platform fees” without specifying what they are doesn’t seem like the best thing. If you charge, or plan to charge, other fees, maybe also mention them on the pricing page.

If you aren’t charging fees yet you can turn it into a promotional thing about how you’re waiving the fees for the first X months.

It seems like this case has nothing to do with the DMCA. Google’s content ID and claim system is not related to that at all.

Maybe the NuXT: https://monotech.fwscart.com/NuXT_v20_MicroATX_Turbo_XT_10MH...