And for the record I’m not for this, but my concerns are more about what various governments may start mandating as this capability becomes an option.
If you want on-device and cloud backup of data that isn’t checked for illegal content, I think that change needs to happen at the legislature not the phone store.
I don’t see this move from Apple making any measurable difference in how well a government can scan your device for arbitrary data.
If it happens it was going to happen anyway, your prediction of the future comes true with or without CSAM scanning, if you are allowing for new government orders and legislation.
This is Apple creating a mechanism for worldwide surveillance and declaring they'd never compromise because they're good people who we can trust. Note that they're not making any actual promises that might land them in court if the system is abuses, they're just repeating that they're good people who'd never cooperate with authoritarian regimes.
- Get your image added to the list
- Order Apple to scan all images not just ones being uploaded to iCloud
- Order Apple to change the reporting threshold or remove the safety voucher system unless you only care about users with many matching images
- Compromise or order Apple to skip the human review and pass the reports directly to your government agency
Really you have to compromise or order Apple to change every step in the pipeline except “compare images to hashes” which is the part an intern could do in a weekend.
I’m sure governments would love to know which users have “undesirable” images on their device. I’m just saying instead of doing the above steps to take advantage of the CSAM system, they could just order Apple to scan for the images they want scanned for. Apple’s choice is the same, comply or risk participation in that market.