Apple's child protection features spark concern within its own ranks: sources
reuters.com
reuters.com
Of course this is going to spark concern within their own ranks. It's like working for a food company that claims to use organic, non-processed, fair-trade ingredients and in just a day deciding that you're going to switch to industrial farming sourcing and ultra-processed ingredients.
It's a complete change of narrative and there's no easy way to explain it and still defend Apple's Privacy narrative, wihout doing extreme mental gymnastics.
That is, it seems like Apple really wanted to preserve "end-to-end" encryption, but they needed to do something to address the CSAM issue lest governments come down on them hard. Thus, my guess is, at least at the beginning, they saw this as a strong win for privacy. As the project progressed, however, it's like they missed "the forest for the trees" and didn't quite realize that their plan to protect end-to-end encryption doesn't mean that much if one "end" is now forced to run tech that can be abused by governments.
This kind of thing isn't that uncommon with large companies. To me it seems very similar to the Netflix Qwikster debacle of a decade ago. Netflix could very clearly see the writing on the wall, that the DVD-by-mail business needed to morph into a streaming business. But they fumbled the implementation, and what largely came across to the public was "you're raising prices by 40%."
I give Netflix huge props for actually reversing the decision pretty quickly - I find that exceptionally rare among large corporations. Remains to be seen if Apple will do the same.
You would think apple would get ahead of this story and mention … or maybe they don’t have any E2E plans at all.
iMessages are already E2E encrypted, but you are correct, iCloud backups are decryptable with a warrant (and that was reportedly added at the FBI's request). But I agree with Ben Thompson's point in that blog post, that it's OK to not have strong, unbreakable encryption be the default, and that it's still possible to use an iPhone without iCloud and get full E2E.
But with Apple's CSAM proposal is NOT possible to have an iPhone that Apple isn't continuously scanning.
As currently implemented, iOS will only scan photos to be uploaded to iCloud Photos. If iCloud Photos is not enabled, then Apple isn't scanning the phone.
I think the issue is that, as Ben Thompson pointed out, the only thing preventing them from scanning other stuff now is just policy, not capability, and now that Pandora's box is open it's going to be much more difficult to resist when a government comes to them and says "we want you to scan messages for subversive content".
I could maybe understand the new implementation if Apple had announced they'd also be enabling E2E encryption of everything in iCloud, and explained it as "this is the only way we can prevent CSAM from being stored on our servers."
Why is that supposed to be their responsibility?
Given the scale of their business, there is effectively a certainty that people are transmitting CSAM via Comcast's network. That's no excuse for them to ban end to end encryption or start pushing out code to scan client devices.
When you hail a taxi, they don't search you for drugs before letting you inside.
Because they're not the police. It isn't their role to enforce the law.
They (and Google, Microsoft, Facebook, etc.) are essentially mandated reporters; if CSAM is on their servers, they're required to report it.
It's like how a doctor is a mandated reporter regarding physical abuse and other issues.
Because they're not the police. It isn't their role to enforce the law.
They're not enforcing the law; if the CSAM reaches a certain threshold, they check it out and if it's the real deal, they report to National Center for Missing and Exploited Children (NCMEC); they get law enforcement involved if necessary.
I don't think that's correct. My understanding is that if they find CSAM, they're obligated to report it (just like anyone is). I don't believe they are legally obligated to proactively look for it. (It would be a PR nightmare for them to have an unchecked CSAM problem on their services, so they do look for it and report it.)
Consider that Apple likely does have CSAM on their servers, because they apparently don't scan iCloud backups right now. I don't believe they're breaking any laws, at least until and unless they find any of it and (hypothetically) don't report it
Quite honestly, my opinion is that any service not scanning for CSAM is living on borrowed time.
Scanning on their servers is much less privacy preserving for users but don't fret, Dropbox, Facebook, Microsoft, etc. already scan stored photos.
Now though, Apple is already saying "We'll take this database of illegal image hashes provided by the government and use it to scan your phone." It's now quite trivial for a government to say "We don't have a special database of just CSAM, and a different database of just Winnie the Pooh memes. We just have one big DB of 'illegal images', and that's what we want you to use to scan the phones."
Have been whining about big data for and ML recognition systems for years.
At last in Apple's case, ML is used only if a minor child (less than 13 years old) who is on a Family account where the parent/guardian has opted-in to the ability to be alerted if potentially bad content is either sent or received using the Messages app.
Because it’s something to push back against governments by saying « I can’t ». But it’s a societal issue if a corporation can say « I don’t want » to a government.
It’s really not the same thing and Apple just burned their « I can’t » card and are implying they can just say « no » to governments. Which is quite an even more dystopian thing.
The statement from Apple is that they will say no. Whether that actually ever happens is something we will see, one way or another.
Do you really think they would care in the slightest about banning iPhone?
Or that Putin cares more about people not being able to buy what amounts to a luxury item, than being able to hunt down opposition supporters?
Ohh... oops.
Any chance this is already operational there?
And, in all honesty, there's a lot of those.
Sounds like political suicide on either side: government interference with the country's largest company, an iconic brand, for no reason other than to hopefully spy more on your citizens.
I can guarantee that some industry bigwigs are salivating at the prospect of this tech. Imagine youtube copyright strikes but local to your device.
That really depends. And is quite a strange, perhaps worrying, take.
Because, for many people, copyright strikes are the least of their problems.
Not that I disagree on it being an issue.
Democratic countries are much more of a danger because Apple has offices in them and cannot easily pull out of the market if they feel the laws are unconscionable.
(Yes, I know, it sounds like I'm splitting hairs distinguishing between copyright maximalism and copyright trolling. Please, humor me.)
What copyright maximalists really want is an end to the legal protections for online platforms (DMCA 512 safe-harbor) so they can go after notorious markets directly. They already partially accomplished this in the EU. It's way more efficient from their end to have one throat to choke. Once the maximalists get what they want, platforms that want to be legitimate would be legally compelled to implement filters, ala YouTube Content ID. But those filters would apply to content that's published on the platform, not just things on your device. Remember: they're not interested in your local storage. They want to keep their movies and music off of YouTube, Vimeo, Twitter, and Facebook.
Furthermore, they largely already have the scanning regime they want. Public video sites make absolutely no sense to E2E encrypt; and most sites that deal in video already have a content fingerprinting solution. It turns out it was already easy enough to withhold content licenses to platforms that accept user uploads, unless they agreed to also start scanning those uploads.
(Side note: I genuinely think that the entire concept of safe harbors for online platforms is going to go away, just as soon as the proposals to do so stop being transparently partisan end-runs around the 1st Amendment. Pre-Internet, the idea that an entity could broadcast speech without having been considered to be the publisher of it didn't really exist. Publishers were publishers and that was that.)
But bad people will always have power.
Prior to motorised vehicles and the telegraph for example, borders were almost impossible to enforce on the general population without significant leakage.
Ironically in spite of powered flight, freedom of movement is in a sense significantly less than even 200 years ago when it comes to movement between countries that do not already have diplomatic ties to one another allowing for state authorized travel.
Incidentally, this is part of why many punishments in the medieval age were so extreme - the actual ability of the state to enforce the law was so pathetically limited by today's standards that they needed a strong element of fear to attempt to control the population.
Borders in medieval Europe weren't nearly as necessary because the physical ability to travel was limited. Forget traveling from Germany to France - just going to the next town over was a life-threatening affair without assistance and wealth. Legally speaking, most peasant farmers were property of their lord's manor. But practically speaking, leaving the manor would be quite difficult on your own.
Many churches railed against motor vehicles because the extra freedom of movement they made possible also broke sexual mores - someone might use that car to engage in prostitution! You see similar arguments made today about birth control or abortion.
Prior to the Internet, American mass communication was effectively censored by the government under a series of legally odd excuses about public interest in efficiently-allocated spectrum. In other words, this was a technical limitation of radio, weaponized to make an end-run around the 1st Amendment. Getting rid of that technical limitation increased freedom. Even today, getting banned from Facebook for spouting too much right-wing nonsense isn't as censorious as, say, the FCC fining you millions of dollars for an accidental swear word.
Whether or not a technology actually winds up increasing or reducing freedom depends more on how it's distributed than on just how much of it there is. Technology doesn't care one way or the other about your freedom. However, freedom is intimately tied to another thing: equity. Systems that economically franchise their subjects, have low inequality, and keep their hierarchies in check, will see the "technology dividend" of increased freedom go to their subjects. Systems that impoverish people, have high inequality, and let their hierarchies grow without bound, will instead squander that dividend for themselves.
This is a feedback effect, too. Most technology is invented in systems with freedom and equality, and then that technology goes on to reinforce those freedoms. Unequal systems squander their subjects' ability to invent new technologies. We didn't see this with Nazi Germany, because the Allies wiped them off the map too quickly; but the Soviet Union lost their technological edge over time. The political hierarchy they had established to replace the prior capitalist one made technological innovation impractical. So, the more you use technology to restrict, censor, or oppress people, the more likely that your country falls behind economically and stagnates. The elites at the top of any hierarchical system - aside from the harshest dictatorships - absolutely do not want that happening.
Capitalism is completely compatible with hierarchies and censorship - indeed one could argue that capitalism is completely incompatible with a true flattening of hierarchies since it rests on the ability of an owner class holding a monopoly over the means of production. The majority of dictatorships around the world use capitalism as the basis of it's economy. Following the dissolving of the USSR, Russia continues to be authoritarian and arguably more so than the USSR was past the Stalin era.
Aside from that, I think it's a little premature to frame the internet's ultimate effect on the world as reducing the ability of the government to censor the population when it's only been around for less than 30 years and we are already seeing mass adoption and development of both censorship and surveillance tech that goes beyond even the wildest dreams of 20th century era dictators.
I don't really buy the argument that most technology is invented in systems with freedom and equality either. It just sounds more like something we want to believe than something borne out by data. The internet and rocket ships were the product of the military - an institution that has more to do with using force to enact the will of it's host nation on others and limiting their freedoms than preserving the freedoms of their own, especially for superpowers like the USA and the CCP, which are effectively immune to conquest by military force.
This is in fact the same for the Silicon Valley and most private industry, you only think all this tech is the product of your freedom and equality because all of the actual extreme inequality and lack of freedom is kept compartmentalised to the global south through long and convoluted supply chains. It's not really freedom if only 10% of the actual people involved in the sustaining of an economy have any semblance of it -leaving aside the observation that for even this 10% that represents the average US citizen, their actual democratic agency in the state or in their job is vanishingly low.
First, the Soviet Union didn't have a prior capitalist hierarchy; the whole reason for Leninism as such was that Russia was a feudal, agrarian society, which in Marx's theory had not progressed to the stage of capitalism and therefore could not progress to communism.
Second, food shortages and mass poverty did not end with the establishment of the Soviet Union; in fact, they became enormously worse. Even before being invaded by Germany in WWII, the Soviet system caused the Holodomor, a famine unprecedented in the history of the Ukraine.
Third, the internet has been around for 52 years, not less than 30. I've personally been using it for 29 years, and I can tell you that it already had a long history when I started using it. One of the founders of Y Combinator first became well-known as a result of breaking significant parts of the internet 33 years ago, an event which resulted in a criminal trial.
Fourth, the internet was the product of universities, although the universities were funded by ARPA. Rocket ships have a long evolution including not only militaries but also recreational fireworks, Tipu Sultan of Mysore, Tsiolkovsky, Goddard, the peaceful space agencies, H. G. Wells, and possibly Lagâri Hasan Çelebi.
Fifth, I don't think the argument is that the technology is necessarily produced by systems with freedom and equality, but that it is invented by them. This is somewhat dubious, but not as open-and-shut wrong as your misunderstanding of it. Goddard's rockets were put into mass production as the V2 in Nazi Germany using slave labor, but he invented them at WPI and Princeton. Tsiolkovsky lived in the Czar's Russia and then the USSR, and his daughter was arrested by the Czar's secret police, but he himself seems to have had considerable freedom to experiment with dirigibles and publish his research (but no slaves to build rockets for him), and indeed he was elected to the Socialist Academy.
I think we can make an excellent case that certain kinds of intellectual repression, whether grassroots or hierarchical, fall very heavily on the kinds of people who tend to invent things. William Kamkwamba's family thought he was insane, Galileo spent the last years of his life under house arrest, Newton was doing alchemical research that could have gotten him burned at the stake in Spain, Qin Shi Huang buried the Mohists alive, Giordano Bruno was in fact burned at the stake, and the repression of Lysenko's intellectual opposition was a major reason for the USSR's and PRC's economic troubles in the 01950s and 01960s.
Living in the so-called "global south" (a term which I have come to regard as useless for understanding the world system, if not actively counterproductive) I have to tell you that there's very little production of advanced technology going on here. But I live in Argentina, and the situation is different in different countries; Indonesia, Thailand, Vietnam, etc., have all done significant technical production for the world economy while in the grip of dictatorships, even though Argentina never has. But most countries don't. If tantalum cost ten times as much, we'd still have cellphones, and you probably wouldn't even be able to detect the price difference.
Something strikes me as odd; Why do you say “Russia” but not “Germany”? Why not say “Soviets” or something similar?
Is it because Russia is still portrayed as the bogeyman in the Anglosphere whereas the Germans are cool now?
But what people are afraid of is how little it takes when the technology is there. And how Western governments seem to morph towards that. And how some bastions of freedom are falling
Even a Jew stuck in a ghetto in Nazi Germany enjoyed significantly better privacy at home from both the state and capital than a wealthy citizen in a typical western liberal nation today.
Soviet Russia and Nazi Germany in fact prove the exact opposite of what point you think you're making. They were foremost in using the "magic tech" of their day - telecommunications and motorised vehicles.
Even then, they had several rebellions and uprisings that were able to develop thanks to lacking the degree of surveillance tech found today.
This is incorrect.
Apple has been saying—since 2019![0]—that they can scan for any potentially illegal content, not just images, not just CSAM, and not even strictly illegal.
That's what should be opposed. CSAM is a red herring.
[0] https://www.macobserver.com/analysis/apple-scans-uploaded-co...
1) The list of CSAM hashes is that provided by the relevant US government agency, that is it.
2) The project is not targeted to roll out anywhere other than the USA.
3) The hashes are baked into the OS, there is no capability to update them other than on signed and delivered Apple OS updates.
4) Apple has exactly the same leverage with EvilGov as at any other time. They can refuse to do as asked, and risk being ejected from the country. Their stated claim is that this is what will happen. We will see.
https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
For example here is a broken PSI protocol in terms of point 3. I don’t think normally in PSI this is considered broken because the server knows the value so it is part of its private set.
Server computes M_s = g . H(m) . S_s
where g is a generator of an elliptic curve, H(m) is the neural hash of the image and S_s is the server blinding secret.
The client computes M_sc = M_s . S_c where S_c is the client ephemeral secret. This M_sc value is the shared key.
The client also computes M_c = g . H(m) . S_c
and sends the M_c value to the server.
The server can now compute M_cs = M_c . S_s = M_sc since they both used the same H(m) values. This allows the server and client to share a key based on the shared image.
However, what happens if the client does it’s step using the ‘wrong’ image. If 3) is to hold it should not be possible for the server to compute the key.
Client computes:
M_sc = M_s . S_c
M_c = g. H(m’) . S_c
Now server computes: M_cs = M_c . S_s = g . H(m’) . S_c . S_s
The secret shares don’t match. But if the server knows H(m’) it can compute:
M_cs’ = M_cs . inv(H(m’)) . H(m)
and this secret share will match
Normally this client side list in PSI is just used to speed up the protocol so the server does not have to do a crypto operation for every element in its set. It is not a pre-commitment from the server.
Also, maybe the way I’m doing it here is just normally broken because it is not robust against low entropy inputs to the hash function.
I've also reversed some of apple's non-public crypto that is used in some of it's services and they have made dubious design decisions in the past they have created weird weaknesses. Without knowing exactly what they are doing I would not try and infer properties that might not exist or trust their implementation.
I think it's hilarious that one of the world' great nations would scan for exactly this.
However, the difference between having a feature enabled based on the value of a seemingly-unrelated user-controlled setting, versus having that feature enabled all the time... is basically zero. Additionally, extending this feature to encompass other kinds of content is a matter of data entry, not engineering. That's the policy angle.
When you don't yet have a capability, it might take a lot of work and commitment to develop it. But on the contrary, policy can be changed with the flip of a switch.
Except for the "oops, due to an unexpected bug in our code, every image, document, and message on your device was being continuously scanned" mea culpa we will see a few months after this goes live.
1. About 3 years ago, there were empty files taking up storage on my iCloud account, and they weren't visible on the website so I couldn't delete them. All it showed was that an excessive amount of storage was taken up. Apple advisors had no idea what was going on and my entire account had to be sent to iCloud engineers to resolve the issue. They never followed up, but it took months for these random ghost files to stop taking up my iCloud storage.
2. Sometimes flipping them on and off a lot causes delays in the OS and then you have to kill the Settings app and re open it to see if they're actually enabled. Back when ATT was just being implemented, I noticed it was grayed out on my phone every time I was signed in to iCloud, but was completely operational when I was signed out. Many others had this issue and there was literally no setting to change within iCloud; it was a bug that engineering literally acknowledged to me in an email (they acknowledged that it happened for some users and fixed it in a software update).
Screwups happen in an increasingly complex OS and I just feel that there will be a day when this type of bug surfaces in addition to everything that already happens to us end users.
OK, but what if Apple silently pushes out an update (or has existing code that gets activated) that "accidentally" sets that number to zero? Or otherwise targets a cryptographic weakness that they know about because they engineered it? That wouldn't require "significant modification".
Funamentally, it's closed software and hardware. You can't and shouldn't trust it. Even if they did do some "significant modification" how are you going to notice/prove it?
I see this number very quickly getting set to '1', because the spin in the opposite direction is "What, so you're saying, people get X freebies of CSAM that they can store in iCloud that Apple will never tell anybody about?"
_That_ is a whole other PR disaster.
After they introduced AirTags I went and disabled item detection on all my devices. Yesterday I went into settings and guess what, that feature is enabled again. On all my devices! I’m not sure when that happened, but my guess would be that latest iOS update caused that…
My point is, you will only have things working to the extent you have testing for, and test coverage is likely less robust for less popular features or where perception tells developers that “no one is going turn that mega-feature off”.
Only images that match the hashes of the database of CSAM held by the National Center for Missing and Exploited Children (NCMEC) that are uploaded to iCloud Photos are checked.
Based on their technical documents, it's not even possible for anything else to be checked; even if they could, they learn nothing from documents that aren't CSAM.
Incorrect. All files on the phone will be checked against a hash database before being uploaded to iCloud. Any time before, which means all the time, if you have iCloud enabled.
A cryptographic safety voucher is created for each photo as they're uploaded. The iPhone doesn't know wether or not anything matched. Nothing happens unless the user reaches a threshold of 30 CSAM images that have been uploaded to iCloud Photos.
Communication safety in Messages is only available for accounts set up as families in iCloud. Parent/guardian accounts must opt in to turn on the feature for their family group. Parental notifications can only be enabled by parents/guardians for child accounts age 12 or younger.
https://www.apple.com/child-safety/pdf/Expanded_Protections_...
If they suddenly start to break you trust - who knows what they will push to that black box next time ? You have no way to tell or prevent that, other than leaving the whole platform altogether.
We don't actually know what is implemented. We only know what Apple's PR machine is saying about it, or do we have the source code somewhere?
There's a very good reason Signal (and previously Whatsapp) were recommended over iMessage.
Not to mention the inability to swap iMessage off for SMS which has made it a favourite exploit target for hacking firms like NSO.
They don't have the keys, so how exactly would they decrypt these messages?
I disagree completely on this. For one, users aren't aware that using iCloud means that Apple has your decryption key and can thereby read and share all of your phone's data.
And two, opt-out is a dark pattern. Particularly if you surround it with other hurdles, like long click-wrap and confusing UI, it's no longer a fair choice, but psychological manipulation to have users concede and accept.
Third, as it's hinted, smarter criminals, the ones the FBI should actually worry about, will know to opt-out. So instead the vast majority of innocent users have their privacy violated in order to help authorities catch "dumb" criminals who could have very well been caught through countless other avenues.
disclaimer: I just read Bruce Schneier's "Click Here To Kill Everybody" after the pipeline ransomware attack, and these points come straight from it.
I guess some people just see dark patterns where I don’t.
Making things "opt-out", without even making it hard, dramatically increases the number of people who opt-in. It can be used for many reasons.
For example, the UK switched it’s organ donation laws from opt-in to opt-out.
Another example, my government has decided that them selling their citizen's personal information to garages, insurance companies, etc, when we buy cars, should also be opt-out.
So they freely sell the car make, acquisition date, buyer's name,… [1][2]
Unless it has changed with GDPR. I’ve never bought a car.
While we're on the topic of GDPR, that is exactly why it insists on making cookies, tracking, and the "sharing of information with partners" opt-in.
And that is without hiding what you’re doing, making it unclear or confusing, or requiring multiple actions to change it.
(Links in French, sorry)
[1]: https://mobile.interieur.gouv.fr/Repertoire-des-informations...
[2]: https://www.carte-grise.org/actus/2014/05-12-L-Etat-vends-le...
We have, yet oddly, it's still called "organ donation".
Because most often, when there’s even the sliver of a doubt on the deceased one's wishes, the corpse will keep all it’s organs to the grave / incinerator.
How is my will involved here?
Which is the bare minimum if you want others to be able to even consider respecting it.
The default choice being, in the absence of any information, the one that increases chances of survival of actually living humans doesn’t seem misguided to me.
——
If you want compulsory and overreaching (although not necessarily bad. Got no opinion on that aspect) rules, over here we:
- can’t disinherit a descendant. The state decides the minimum each one of your genetic or legal relatives can get from your assets
- have to financially support our adult children until an arbitrary time decided by the law
- have to support and assist our parents and grand-parents if they can’t provide for themselves
- basically can be legally compelled to financially assist any ascendant or descendant
- may end up having to pay up after a divorce, even if you had an iron-clad contract, to compensate for the loss of QoL of the spouse with a lesser income
... and if you do not know that an opt-out is required to not have your organs removed, then how can it be said that you have willingly contributed them. And if your organs are removed by default, without you actually willing that this be the case, then how can this practice be called "donation"?
So instead, shall we call it "organ retrieval", "organ harvesting" and so on? All rather ugly words. But when the words that accurately describe what you are doing appear ugly, instead of looking for words which inaccurately describe what you are doing, perhaps you should ask yourself if you should be doing what you are doing.
And I'd be interested to know of the percentage of UK adults who are aware of the opt-out. I had a search around but couldn't find any surveys, are you aware of any?
"Do you want to enable iCloud photos? Your private photos will be uploaded encrypted to Apple's servers, so that only you can access them. Before uploading, your photos will be locally scanned on your device for any illegal content. Apple will only be notified once a sufficient volume of illegal content is detected."
An option like this, after the lawyers get at it, will be 14 pages long.
"We can change this at any time", "except by court order", "no guarantee", "not liable", and on and on and on...
This would be a large change from the way it works today, where Apple can view your iCloud photos. They’ve not made any statements indicating that is going to change.
It's fine if you know you want to be the sole person in the world who can get to your data and have the discipline to store master passwords, and you accept Apple support cannot help you even if you have a purchase receipt for the device. But for the average older person that's not a good default.
Apple just needs to design good UX to help these people.
Perhaps a large cardboard 'key' comes with every phone. When first setting up the phone, you scan a QR code on it to encrypt your data with the key, and then you tell the owner that of they ever lose the key to their phone, they won't be able to get in again.
People understand that - it's like losing the only key to a safe.
From time to time you require them to rescan the key to verify they haven't lost it, and if they have, let them generate a new one.
I really like E2E encryption of messages with proper backups on my end. It's good to know the carrier of my messages can't scan them for advertising purpose. But for my phone if I know there is no way to easily get the data back if I have an issue that seems like a major hassle with very little upsides.
It's better if the key is generated when you already have the phone.
For even better security they can instead provide an NFC smartcard or hybrid NFC + USB (like a Yubikey) where the key is stored in hardware and can't be extracted.
Everything about Apples messaging makes you believe otherwise. That seems pretty disingenuous.
Then again, 99% of consumers have very little choice that doesn’t include huddles and complicated setup. We all get the same moon goo, under a differ different brands.
Good, bad, or just the fact of life?
iCloud data is encrypted at rest (edit: except for Mail apparently). The type of encryption (service or end-to-end [E2E]) is specified here: https://support.apple.com/en-us/HT202303
It can be argued that from a user's viewpoint not having E2E encryption is tantamount to not having encryption at all, but from a technical standpoint the data is encrypted.
According to the table on the second link iCloud Photos are encrypted on the server (at rest). Am I missing something?
Apple (and thus, LEO) absolutely can look at your photos on iCloud. What you are missing is that "encrypted at rest" is essentially "not encrypted in any meaningful way".
Edit: I also meant iCloud backups in my original post and how Apple can decrypt your E2E encrypted iMessages with the key the backups contain. But I posted it last night and couldn't edit it once I caught the error. It would be amazing for other iCloud services to have E2E encryption so long as the implications of iCloud backups having your encryption keys is stated front and center when choosing to opt-in.
Unsophisticated users who have lost or forgotten their passphrase can recover their data.
Sure it is—just don't use iCloud Photos. They've been quite clear about this.
Even with iCloud off, the other endpoint (the phones you're iMessaging with) will be leaking your conversations to Apple because they will still have iCloud Backup defaulted to on.
iMessage is no longer an e2e messenger, and Apple intentionally preserves this backdoor for the FBI and IC.
They access 30,000+ users' data per year without warrants.
This is mistaken. If you turn off iCloud sync, Apple isn't continuously scanning your phone.
Not to be nit-picky but IIRC, this isn't exactly how it went down. icloud backups have always been decryptable and Apple had announced they were planning on changing this and making them fully encrypted but when pressured by the FBI, they scraped those plans and left them the way they are.
Unless you use iCloud Backup for iMessage, in which case Apple holds the keys to decrypt them.
This is not accurate. Scanning doesn’t happen without iCloud photos enabled.
You can’t turn that off!!
There’s no way to tell the iPhone, “stop with the AI recognition on my photos.”
Since Apple has decided it’s going to categorize all your photos based on AI, it was just a matter of time before they started categorizing some bad stuff.
As far as encrypted backups go, it's an open question whether they want to deal with the legal and support headaches that such a change would bring. If they continued to do nothing, Congress might force their hand by legislatively outlawing stronger encryption - they had to shit or get off the pot.
For users, if you enable this feature, but then lose your password, you are entirely screwed and Apple can't help you. Encrypting "In-transit" as a middle ground is likely good enough for most people, until researchers manage to come up with a better solution.
Exactly the same with the phone if you forget your PIN/passcode. So they already do this.
The amount of money that is spend on their devices is just insane.
Second, my comment was meant in the context of iCloud - if you don't enable the feature as described, Apple may be able to decrypt and recover something, but if you do go all the way to the logical end, yes, you're out of options.
iOS 15 beta has new recovery option by secret (which is useful only on E2EE?)
And Child Safety was released most likely because the leaks. Motive of the leaks is unknown. They might be waiting for September now.
For content that is pirated Apple would receive a slap on the wrist, but for content that shows the exploitation of children? And that Apple is hosting on their owned servers? There is no doubt every government will throw their full legal weight behind that case.
Now they are stuck between a rock and a hard place. Do they or do they not look at iCloud Photos? Well, not in the cloud i guess, but right before you upload them. Is that any better? I don't know? Probably not. They really have no choice though. They either scan in the cloud of scan on the device. Scanning in the cloud would at least let people know where their content truly stands, but then they can't advertise end-to-end encryption and on paper it may have felt like a bigger reversal.
I think the long and short answer is. Apple cannot really legally protect your privacy when it comes to data stored on their servers. The end.
Apple should use it's gigantic legal arm to set good precedents for privacy in court.
The lawyerly phrase is “knew or should have known.” It could be argued that now that this technology is feasible, failure to apply it is complicity.
Think about GDPR. Eventually homomorphic encryption is going to be at a point where “we need your data in the clear so we can do computations with it on the server” will cross from “failure to be on the bleeding edge of privacy research” to “careless and backwards.”
Pull all apple products from the shelves for a month and launch a billion dollar ad campaign about what the hell our government is up too.
At least try to stick to one governments. Tech companies are entering tricky waters by trying to follow laws of many nations (which conflict) simply because their customers are there
And nothing stops Apple from fighting against the legislative assault. They have made themselves a champion of user privacy and have deep pockets to fight the fight, yet they just surrender at first attempt.
> surrender at first attempt
The irony
When I was young, I was not effectively spied on. My activity and digital trail were ephemeral, and hard to pin down. i wasn't the subkect of constant analysis and scrutiny. I had the privilege of being invited into the engine of an idling train to share an Engineer's drinks along with my parents and no one batted an eye.
There were pedophiles then too. guess what? We didn't sacrifice everyone's right to privacy. We didn't lock more and more rights out of the reach of minors. You didn't need a damn college degree and an insane amount of idealistic commitment to opt out of the surveillance machine. I could fill most prescriptions, even out of State.
We didn't fear the monster. They were hunted with everything we could justify mustering, but you weren't treated as one by default.
I imagine that maybe, somehow, tomorrow's youth may get to experience that. Not live in fear, but in appreciation of the fact that we too, looked into things absolutely worth being afraid of, but resisted the temptation to surrender yet more liberties for the promise of a safety that will never manifest. Just change its stripes.
Those are the best nights.
No. My elders and society just denied the monsters existed. And punished any one who talked about it.
I know they’re out there in the world, but I leave my home anyways, because living my entire life curled up in my bed in fear would be a sad and short life.
I’m more afraid that people will lose the ability to be happy because they’re so full of fear and skepticism that they don’t form meaningful relationships, relax, experience life.
I’m also more afraid of what can happen when a small group of people are allowed to dictate and control the lives of many, justifying it by pushing fear. Then it’ll be more than a small group of abusers harming a small group of people (in this case, child abusers). It becomes a different group of abusers (dictators) harming the entire population through their dystopian politics.
How many people, including children, died when a group of dictators used fear & skepticism in the 1930s to push an agenda that ultimately led to WWII and concentration camps?
We need to be extraordinarily careful that we don’t try to save a very small number of lives at the expense of many lives in the future, as a result of policies that we put in place today.
They could have announced this all in one shot and saved themselves a lot of blowback.
I believe they have to understand that, which makes me think the theory of them planning to add E2E may not be true.
They would have to say something like "We know some users want to fully encrypt their cloud data. We also want to encrypt their data. But the government won't allow us to do it until we protect their interests in stopping child abuse and terrorism". That would contradict their marketing image of having their interests being aligned with their users'.
Not just one end, but specifically the end that their users thought they had control over.
Sincere or not, the "the only way we could do true X is by making X utterly useless and meaningless" thing certainly seems like bureaucracy speaking. It's a testament to "the reality distortion field" or "drinking the Koolaid" but it shows that the "fall-off" of such fields can be steep.
Consider the this new hypothetical future state:
1) photos are encrypted on iCloud
2) photos are hash scanned before being encrypted and uploaded.
If you don’t upload photos to iCloud there is no change.
In all scenarios you have to trust Apple to write software that does what they say and that they won’t write software in response to a subpoena to target you specifically.
The attack with the hash scanning is someone gets a hash for something in your photos specifically (that doesn’t match a ton of people) and through some means gets it into the database (plausible), but can’t get anything without going though this route. Specifically subpoenas can’t just get all your photos without going through this route.
My conclusion is that you are more protected in the hash scan and encrypt state than in current state.
Ashton Kutcher's foundation Thorn https://en.wikipedia.org/wiki/Thorn_(organization) does a lot of work in this space. They have hashed content databases, they have suspected and watchlisted IP addresses, etc...
I don't know why I'm being downvoted. This is an active area of cryptography research, and I've applied these types of content checks in real life. You can maintain good privacy and still help combat child pornography.
Sometimes HN takes digital cynicism a step too far.
This seems like an incredible reach to me.
And is "pressure from the FBI" really the only reason why Apple might hold backup keys? Please sketch out your "lost device, need to restore from backup" user journey. For a real person, who has only an iPhone.
How about "enter your master passphrase to access your photos". Like literally all syncing password managers work, or things like Authy work to back up your one-time password seeds. It's not complicated.
Occam's razor also says they aren't playing 11th dimensional chess, and that they just fucked up.
First they enable this technology to prove to the government that they can still scan for bad stuff, then they enable true E2E. I don't know if this is really their motivation but it sounds plausible to me.
EDIT: though you are right - they can please governments by making encryption useless while lying to people they still have it. Not sure that was your message though.
It seems plausible to me that Apple might believe some of their privacy conscious users would prefer this situation and hence "privacy" can be considered a plausible explanation for their actions here.
>... to address the CSAM issue lest governments come down on them hard.
And I think that is the springboard of why this is happening.
It's no secret that Apple has lobbyists which by turns have a "pulse" on the trajectory of Bills in the House and Senate.
What immediately came to mind to me was H.R.485 of 2021's House Session:
https://www.congress.gov/bill/117th-congress/house-bill/485/...
My (layperson) estimation is that Apple knows the rewording of it is liable pass both House/Senate in some way either in the 2022 or 2023 sessions and are attempting to get ahead of the issue. (most likely as a "rider" bill at this point)
That's my thought process on what may have precipitated Apple's move toward "CSAM"; and I welcome it in some ways (unequivocally, children should not be subjected to sexual abuse) and am wary of it in others (ML is in its infancy in too many respects, and if a false-positive arises who's to say what's liable to happen on the Law Enforcement level).
There are infinitely many concerns to be discussed on both sides of this argument, and I want both to succeed in their way. CSAM, however, feels like an attempt to throw a blanket over Apple's domains-of-future-interests while playing an Altruism Card.
One thing is certain, though: they've opened the floodgates to these discussions on an international level...
Great, let the bills pass and then they get to publicly blame congress for being "forced to legally comply" instead of jumping the gun and doing an about face on 80% of your reputation built over the last 20+ years.
There is 0 reason to do this before being legally forced to. It's not a gamble with a possible upside somewhere. It's directly shooting yourself in the foot.
If Apple is surprised by the blowback, it’s entirely Apple’s fault because it didn’t engage with many outside experts on the privacy front.
I recall reading either in Alex Stamos’s tweets or Matthew Green’s tweets that Apple was invited before for discussions related to handling CSAM, but it didn’t participate.
Secretiveness is helpful to keep an advantage, but I can’t believe that Apple is incapable of engaging with knowledgeable people outside with legal agreements to keep things under wraps.
This blowback is well deserved for the way Apple operates. And I’m writing this as a disappointed Apple ecosystem user.
If Apple does not have decryption keys for iCloud content, they can't decrypt it, only the user can. For Apple, it's not technically feasible, and the law supports that.
Apple has now demonstrated that it is technically feasible to overcome this challenge by doing things "pre-encryption" on the user's device.
From a user's point-of-view, Apple's privacy is worth nothing. It makes no sense for them to encrypt anything, if they leak all data before encryption. Children today, terrorism tomorrow. All your messages, photos, and audio, can be leaked.
There are thousands of people at Apple involved in the release of any single feature, from devs and testers, to managers all the way up to VPs.
The fact that this feature made it to production proofs that Apple is an organization that I can't trust with my privacy.
Their privacy team must be pulling their hairs out. They better start looking for a new job.
> From a legal stand point, they are now screwed, because now they _must_ do it.
That's not the right takeaway. They're only required to turn over data to which they have access. If they continue writing software that makes it so they don't have access to the data, then they do not need to turn it over because they can't. The fact that you can write software to store or transmit unencrypted data is irrelevant.
I don't argue with your point, but actually Reed Hastings originally created Netflix the DVD rental company so that he could turn it into a movie streaming company. He had to wait for internet to be fast enough. His interview with Reid Hoffman is pretty cool:
It's actually easier to the the scanning on the backend compared to scanning of the user's phones.
My opinion, maybe entirely wrong, is that it's related to how Apple operates in China. In China, Apple does not actually operate their backends, but their BE is operated by government owned business, which is "leased" by Apple.
Maybe they don't want to enable scanning on BE for that reason, and over-engineered local device scanning, that really doesn't make much sense if you think about it twice?
The iMessage thing is actual spyware, by design, it is a way for parents to spy on their children, which is fine as long as it is a choice and the parents are in control. And that's the case here.
The iCloud thing is much more restrictive, comparing hashes of pictures before you send them to Apple servers. For some reason, they do it client side and not server side. AFAIK, iCloud photos are not E2E encrypted so I don't really know why they are doing it that way when everyone else does it server-side.
But combine the news and it sounds like Apple is using AI to scan all your pictures. Which is not true. Unless you are a child and still under your parents authority, it will only compare hashes of picture you send in the cloud, no fancy AI here, and it is kind of understandable for Apple not to want to host child porn.
I am increasingly convinced this is a backroom deal, just like the iCloud Backup e2e feature (designed and, as I understand it, partially implemented) being nixed on demand (but not legal compulsion) from the FBI.
If you get big enough, the 1A doesn't apply to you anymore.
Apple are scanning the images only when they are uploaded to iCloud, because they don't want the CSAM pictures on their servers.
And you're not alone. Reddit, Facebook, Twitter etc are full of shouting angry people who haven't understood the story at all.
It's almost became a Rorschach test between those who understand technology and those who like to shout at things.
> > Apple really wanted to preserve "end-to-end" encryption, but they needed to do something to address the CSAM issue
There we are.
Maybe they would have been surprised where it ended up, but someone in charge there is definitely on board with a reversal on the previous privacy stance, given the doubling-down with an email signal-boosting the NCMEC calling privacy concerns the "screeching of the minority".
Almost everyone was traced back to the Jan 6th insurrection, because unlike antifa they took their phones with them, so easy trace for government. Now they are in solitary confinement in prison while the world loughs at their expectations of privacy from their phones.
If they get a covid passport implemented, you won't be able to go out in public without your phone to prove you've been vaccinated. So leaving a phone behind will not be much of an option if this is to pass. Metadata and contact tracing will keep track of everyone you're near. This dystopian future is possibly just a few weeks away.
I don't really see how they can recover from this. Will they go out of business? no. Will they even lose market share? No. But there will always be that example of the time Apple got it wrong.
If I was an Apple employee I would certainly be concerned about how something like this could happen.
I just don’t think the messaging they had around this is reassuring at all. While they had all sorts of technical explanations as to how trustworthy this will all be because they are in charge, the whole thing quickly went from “this is only for iCloud photos” and only in the US to “3rd party integration can happen” and “we’re expanding it to other countries”. Which I guess is a logical next step but it is a hint at expansion.
The walled garden becomes much less tempting after all of this, especially right after a scandal like the Pegasus one.
yes, but at least they are telling the truth now...
Apple is saying this: "we need to scan your phone because we need to look for this material, and by the way, we're only scanning things _we've been scanning for years already_"
Basiclaly, everything else aside, in the current state of things, this feature makes no sense. They are not scanning anything they weren't scanning already (literally, everything on iCloud) so why did they bother?
A bunch of people are making speculative claims that Apple is doing this to enable encryption of data in iCloud, but there's no reason to believe that's true. Apple certainly hasn't mentioned it.
So .. why?
There is nothing surprising here. Apple never mentions things unless it’s ready for release or is just about to be released. It’s just how Apple works.
I can hope, I guess...
As far as we know, Apple is not already using their decryption keys to decrypt images upload to iCloud and compare them with known images. That's according to this blog post that claims Apple make very few reports to the NCMEC:
https://www.hackerfactor.com/blog/index.php?/archives/929-On...
I don't quite understand why people are so against this. What if some algorithm scans your images? If you are using any cloud service to share your images then your images are already being scanned and far more invasively. Is the the storing of hashes that concerns you? Your password(s) is already being stored as a hash and we trust that it can not be reversed back to plain text, why wouldn't we trust that these image hashes can't be reversed back into images?
I've seen some notion that this is a backdoor and governments and other Lovecraftian entities could use it to suppress freedom of speech by censoring certain images, but that would require them to first have the image in question, then creating hash of the image, and then inserting it into the Apple's database and after that it would only be flagged for human verification and again assuming you trust Apple to handle rest of your digital life (such as iMessages) why wouldn't you trust Apple to share your images? If some entity has it's hooks so deep in apple that they start to censor images why wouldn't they just stop all of your messages being sent?
Please correct me if I'm missing something obvious.
You should avoid making political or moral arguments when your technical knowledge is 10 feet below your ego. An image is orders of magnitude larger than a password, meaning that many more collisions. Also, because it's a perceptual hash, even more collisions. One does not simply reverse a perceptual hash. Keep your hat on ;-)
I am not being political or moral. Why would you think that? Is this one of those "since you don't immediately agree with me it means you are the enemy and disagree on everything" type things?
>An image is orders of magnitude larger than a password, meaning that many more collisions.
Whole point of hashing algorithms is that they produce unique hashes for different inputs. Wouldn't a larger input value (bits of the image) make it harder to have collisions?
>One does not simply reverse a perceptual hash
So what actually is the privacy concern here? There is no way to get the original image from the hash at best you can get an approximation of it.
>Keep your hat on ;-)
I guess this is suppose to be some kind of dig at me. No need to be an asshole even if you don't agree with someone.
Standard cryptographic hashes should have certain properties like https://en.wikipedia.org/wiki/Avalanche_effect
However, a perceptual hash desires the opposite. Due to the pigeonhole principle, a larger space mapping to a smaller space, involves more collisions. In fact, all hashes have infinite collisions, one just tries to design hashes that don't have many collisions on smaller length inputs. Ultimately though, there will be at least 2^n collisions for all inputs n bits long if the hash is n bit output. You can easily calculate this by looking at the excess in size between input space size and output space size.
https://en.wikipedia.org/wiki/Pigeonhole_principle
Perceptual hashes are essentially designed to collide on similar data. The fine details are lost. An ideal perceptual hash algorithm would quantize as many alterable properties of an image as possible. Contrast, brightness, edges, hue, fine details, etc. In the end, you have a bunch of splotches in a certain composition that form the low dimensional eigenbasis of the hash.
Police knocking down your door will only happen if you are found to posses multiple abuse pictures and they suspect you might be endagering people.
This just feels like people constructing imaginary boogieman out of this
The question really is, will it be used for intended purposes or will it become like roadside drug sniffing dogs? A common euphemism is "probable cause on 4 legs."
https://reason.com/2021/05/13/the-police-dog-who-cried-drugs...
If it is overly sensitive, or if Apple or the government expands the list to include much more than what it has initially been slated for, it's possible that the faux detections will be used for escalation and ultimately result in parallel constructions for crimes like drug possession.
To clarify - here's an arcane scenario. Authorities receive an alert and are given permission by Apple to look at all photos taken the same day as the photo in question. The photo in question is a false match but the rest of the photos that day are still inspected. The agents observe a photo of a bag of white powder, and use it to procure further investigations/warrants. The target is later arrested for possession once physical evidence is found.
https://en.wikipedia.org/wiki/Parallel_construction
I find the laws in this nation, particularly drug laws, to be quite obtuse. I don't think it is wise to give any additional enforcement capabilities to those that are continuing to deprive citizens of life and liberty over archaic drug laws.
I hate pedophiles and child abusers as much as anyone else should. I'd execute the abusers if we are being totally honest. It's just that this technology strikes me as a potentially Orwellian device.
This might not be true, but I've always assumed that if I am suspected of something the police can already access all my iCloud (or any other cloud storage) images with court order.
What court order?
Apple spots the hash collision, snitches on you, and then police get all your photos. Isn't that how this is supposed to work?
They'll ditch privacy in a second if they can sell you enough ads, just as they ditched privacy for Chinese customers when the Chinese government asked them to.
You could spin that the other way: Apple’s foray into advertising was opportunistic, but didn’t sit with their focus on privacy. You’d need internal memos to prove it went either way.
Between that and Apple's principled stance against the FBI, I'm inclined to believe that at that time, they were making principled choices out of a real concern for user privacy.
I suspect this time around they have a real concern for dealing with the growing child exploitation/abuse problem, but failed to prioritize system design principles in their zeal to pursue positive change in the world.
I don't envy them: in their position, they have an incredible amount of pressure and the stakes are incredibly high.
May cool heads and rational, principled thinking prevail.
No it's not, it's more like in just a day they started including a chemical preservative amongst all the organic ingredients. All the good stuff is still there, it's just pointless now for people who want to eat pure organic food.
I agree with the sentiment though.
Once they open the door they will not be able to close it ever.
As it stands, the system is designed so that the phone snitches you out if you have CSAM on it matching some perceptual hashes (and plenty of it), but if you do not have CSAM above threshold amounts then your phone never even phones home about it. It is in fact a privacy-preserving mechanism in the sense that Apple never sees your photos.
https://www.nbcnews.com/id/wbna8161067
* edit: hahaha downvotes thanks apple employees!
Is it supposed to make us feel better the scanning is happening on our own devices instead of someone else’s?
It certainly increases the “creep factor.”
I’m not even talking about any “slippery slope” scenarios and I’ll never have any of the material they are looking for. And right or wrong, I don’t really fear a false identification, so this isn’t about a worry that they will actually turn me in. I just don’t want them scanning my phone for the purpose of turning me in. I definitely do not want to take an OS update or new phone that does this.
(False positives are the huge flaw in this system, though. There are human judgements by anonymous people using opaque processes without appeal throughout this. A lot of damage can be done before anything goes to court, not to mention courts themselves are susceptible to the illusion of infallible technology. Others have well laid out the scenarios that will result in serious consequences for innocent people.)
But the idea of being suspected even in this abstract way, because of something other people do, is at the very least distasteful, bordering on offensive.
This is the word that most captures my feeling not just around the feature but how it was rolled out as well. Reading the PR announcement and then the leaked memo just felt - quite distasteful.
There are a number of things Apple has done and continues to do that fit into that category for me personally - the fact that iOS is so locked down for example.
But this really takes the cake. It's like someone at Apple dialed up the distasteful to 11 and let loose.
The whole industry does distasteful things but this is a harbinger of our inability to trust them at a deeper level. Not just Apple.
Large government to Apple: "Please now also create a hash on each photo metadata field including date/time and location. Let us query these. AND BTW, this change must be kept secret. Thanks."
How do you know this hasn’t already happened? What does it have to do with the CSAM technology?
Exactly like an Invisible Pink Unicorn.
If you go after imaginary bad things, you will never stop. That is the problem with paranoia.
The “UK porn filter” has already been extended to all sorts of “« extremism online »” (to no effect in the capital of knife attacks, it seems — as usual invasive police rights do not equal a reduction of criminality) and it’s already being proposed to be extended to:
- Online Harms
- Online Safety Bill
https://en.wikipedia.org/wiki/Web_blocking_in_the_United_Kin...
It’s that if they do, this technology is irrelevant.
1. Did not exist prior (scanning stuff on the endpoint)
2. Has a plausible abuse case (the same system, applied to stuff that isn't CSAM)
I find this very compelling, especially in the shittier regimes (China...) that Apple operates in.
What stops Apple from altering the deal further?
And if you have an answer for that, what makes you believe that 10 years in the future, with someone else at the helm, that they won't?
Your device is now proactively acting as a stool pigeon for the government where it wasn't prior. This is a new capacity.
And it's for CSAM. For now. The technological leap from locally scanning just the stuff you upload, to scanning everything, is a very simple change. The leap from scanning for CSAM to scanning for some other perceptual hashes the government wants matched is very simple.
If they do, then they do. There are never any guarantees about anyone’s future behavior. However just saying ‘what stops you from becoming a thief?’, does not imply that you will become a thief.
There is no new capability other than what they have said. They already had far more general purpose mechanisms for analysing and searching your files already in the operating system.
The leap from doing spotlight indexing to searching for the keywords ‘falun gong’ or ‘proud boys’ in your files is also simple. So is the leap from searching your photos locally for ‘dogs’, to searching locally for ‘swastikas’ and reporting back when they are found.
If they decide to build some spyware, there is no need for it to be based on this. It’s a red herring.
That’s the problem: Apples promise means nothing exactly because it’s so easy to abuse.
Apple says they will refuse when asked to scan for anything that is not CSAM. That’s one of those nice ‘technically true’ statements lawyers like to include.
Apple will not have to refuse anything, because they won’t be asked.
Apple doesn’t get a huge pile of CSAM images (obviously), they get a list of hashes from a 3rd party. They have no way of ensuring that these hashes are only for CSAM content. And when ‘manually’ checking, they aren’t actually looking at potential CSAM, they are checking the ‘visual derivative’ (whatever that means exactly), basically a manual check if the hashes match.
So yes, they would technically refuse if asked to scan for other material, but they will just accept any list of hashes provided to them by NCMEC (an organization created by the US government and funded by the US DoJ) no questions asked. The US government could include any hash they wish into this list without Apple ever noticing.
The visual derivative is enough to tell the difference between a collection of CSAM and say documents, or pictures of protests.
It can’t be abused without Apple’s involvement.
How would you know ? They have never given details on what this 'visual derivative' actually is. If it's detailed enough to recognise it as CSAM, then Apple isn't allowed to look at it.
However this is moot anyway. Using this mechanism to search for things other than CSAM would be unconstitutional under the 4th amendment.
https://www.economist.com/united-states/2021/08/12/a-38-year...
https://9to5mac.com/2019/01/05/apple-privacy-billboard-vegas...
Selling dev consoles on ebay or drugs on the darkweb? We'll take the picture in the listing and find out who took it.
Can't find any matches? This is too ineffective. We'll need Apple to store all hashes on all devices for 6 months.
The slippery slope argument makes a LOT of sense.
https://www.economist.com/united-states/2021/08/12/a-38-year...
Yeah no, there are about 1b iphones in use, even a ridiculous low amount of false positive would be a major pain in the ass for a lot of people regularly.
The slippery slope argument is totally valid, whatever legal safeguards you have today might not exist tomorrow, but the tech will stay here forever.
I actually fail to see how your second sentence support the first. Treating everyone as a potential suspect of a crime (which isn't even wide spread in the first place) is a major slippery slope. Today it's "for the children and only in the US" tomorrow it might be "find me pro Palestinians pictures of people living in Israel". Nobody can predict what apple stance will be in 25+ years
Yes, they do: https://news.ycombinator.com/item?id=28160092
> For example, if Apple one day decided to detect unauthorized copyrighted media, they could report it to the RIAA. The RIAA is independent of the government, thus the 4th amendment is not violated.
It’s true that Apple could decide to implement any search or scanning mechanism at any time, and report anything they like to anyone they want to. So what? This is true of anyone who writes software that handles user data.
What does that have to do with the CSAM mechanism? It seems like an unrelated fear.
He CSAM mechanism proves that Apple can and will go this route, even if users are against that.
How exactly does it prove that?
How difficult would it be to transform it into looking for images from political dissenters? Now, you have a magical list to find targets to probe further.
Maybe you want to identify behaviors you disagree with.
Images tell a story and maybe not to the people who are looking out for your best interests.
Wrong think always changes.
[1] https://www.apple.com/child-safety/pdf/Expanded_Protections_...
And look at the diagram. The matching of image hashes with user photos occurs "on device."
I just think that there are better arguments against it without unnecessary exaggeration. So I wanted to be more specific on which images on your device are scanned according to that paper.
Obviously I should have phrased is better, like “about to be uploaded to iCloud”…
Just to spell this out (since I think this thread could still be a little confusing):
1. The photo scanning does occur on the device, not in the cloud. However,
2. Apple's explanation indicates that this device-based scanning will only occur on photos that may be uploaded to iCloud.
- as part of the home?
- as part of the human body?
In either cases we’re allowed to do crime at home as long as no-one knows it.
Yeah - this is an architecture designed to be abused.
Yes. This is called vigilantism.[1] Apple is proposing to become a vigilante.
1. https://en.wikipedia.org/wiki/Vigilantism
"Vigilantism is the act of enforcement, investigation or punishment of perceived offenses without legal authority."
They want to have end-to-end encryption and store your photos encrypted. This is why they scan before sending them encrypted.
They won't scan anything which you would not put in the cloud anyway.
This is pretty clever way to preserve end-to-end encryption and satisfy requirement not to store anything CP related on their servers. Probably too clever for journalists to understand :(
Correction: Anything in the shady government-supplied database, pushed through a private corporation, not CP. The official purpose today is CP (well, CSAM, which is already more vague than CP), but the distinction is important. Also, not only to prevent storing that content on their servers, but to report you to the authorities.
It's a pretty clever way to normalize adversarial devices, there's no preserving of any kind of privacy going on here.
I contend that if they are storing encrypted data that they cannot decode, then they are not knowingly storing CP at all.
If someone then downloads that, decrypts it with the key that only they know, and it turns out to be CP, then they are the ones storing CP, which as we all know is already illegal.
Obviously I'm not a lawyer and this isn't how icloud works, but lots of companies legally store encrypted data that they can't decode.
There's no legal requirement that the storage provider needs to ensure there's no CP, only that they don't knowingly store it.
Apple just doesn't want to allow people to have real encryption that they control, almost definitely so they can make more money by forcing people to use expensive icloud instead of local encrypted backups, and retain the ability to scan everyone's data. All under the flimsy excuse that 0.1% of people might forget their password and be unable to unlock their data. Nevermind you could only allow this for power users at their own request but "it's for your own good because you're too dumb to manage a password", lol fuck you Apple.
So it's just yet another self-serving, penny pinching, thinly veiled customer privacy violation, this time wrapped in a big fat "think of the children", which as we know is corporate-speak for "turn brain off now - we want to do $thing_you_wont_like".
I'm yet to think of how Apple plans to make money out of this one, but I'm sure there's an angle in there somewhere and we'll find out eventually.
Maybe simply that either NSA or China said "give us this new scanning system or we shut you down" - and I'm not American but I do wonder about people who still cling to the belief that the fourth amendment would be respected post Snowden revelations.
Hell, I'm Australian - they're probably just planning to push that hash-matched data through servers in Australia so they can see everything, because Australia has no bill of rights, and getting more authoritarian with every uncontested bill that passes. Or some other kind of "legal" fuckery to move even closer to the dystopia they are so hell-bent on creating.
Or their TV scans every frame displayed and every word said if it sees anything the TV manufacturer thinks is illegal it's reported to the police?
Or their blender has an array of chemical sensors in it, and if it detects a substance the manufacturer thinks is illegal it's reported to the police?
How about smart floorboards that detect illegal dancing by women in Iran? Shall we do that too?
Apple where are you on that one? I can point you to a country filled with eager customers, seems right up your alley.
FYI most (all?) modern printers will refuse to print anything they think is counterfeit money, and will include a unique code traceable back to you in every printed document. - https://en.wikipedia.org/wiki/Machine_Identification_Code
I still think it's BS that you can't scan money though and it very much an overreach. Your home printer isn't gonna make a convincing fake anyway.
Let's also remember that malware would easily drop illegal images as a way to mess with the system and specific people.
If they scan my iCloud photos in iCloud, well lots of companies scan stuff when you upload it. It's on their servers, they're responsible for it. They don't want to be hosting CSAM.
It feels much worse them turning your own, trusty iPhone against you.
I know that isn't how you should look at it, but that's still how it feels.
They have no business scanning the phones for anything, period.
I'll refer you to the poem beginning with "First they came for the Jews...".
Als die Nazis die Kommunisten holten, habe ich geschwiegen;
ich war ja kein Kommunist.
Als sie die Sozialdemokraten einsperrten, habe ich geschwiegen;
ich war ja kein Sozialdemokrat.
Als sie die Gewerkschafter holten, habe ich geschwiegen;
ich war ja kein Gewerkschafter.
Als sie die Juden holten, habe ich geschwiegen;
ich war ja kein Jude.
Als sie mich holten,
gab es keinen mehr, der protestieren konnte.
This all plays so perfectly into my long-time fears about the locked down nature of the iPhone. Sure, it's more secure if Apple is a good steward, but no one outside of Apple can inspect or audit what it's doing!
The only difference is intent. The technical risk has not changed at all.
And not just Apple. Once technical feasibility is proven, I can easily see governments mandating this scheme for all devices sold. At that point, it can get even more ugly, since e.g. custom ROMs and such could be seen as a loophole, and cracked down upon.
I will also note that, while it may be a hypothetical in this particular instance as yet, EU already went from passing a law that allows companies to do something similar voluntarily (previously, they'd be running afoul of privacy regulations), to a proposed bill making it mandatory - in less than a year's time. I don't see why US would be any different in that regard.
At this point of the discussion, people usually pivot to scope creep: the on-device scanning could scan all your device data, instead of just the data you put on the cloud. This claim assumes that legislators are too dumb to connect the fact that if their phone can search for dogs with “on-device processing,” then it could also search for contraband. I doubt it. And even if they are, the national security apparatus will surely discover this argument for them, aided by the Andurils and NSOs of the world.
As I have repeatedly said: the reaction to this announcement sounds more like a collective reckoning of where we are as humans and not any particular new risk introduced by Apple. In the Apple vs. FBI letter, Tim urged us to have a discussion about encryption, when we want it, why we want it, and to what extent we should protect it. Instead, we elected Trump.
The precedent now established by Apple is that it's okay to scan things that are physically in possession of the user. Furthermore, they claim that they can do it without actually violating privacy (which is false, given that there's a manual verification step).
Apple’s claim, further, is that this approach is more privacy-preserving than one which requires your cloud provider to run undisclosed algorithms on your plaintext photo library. They don’t say this is not “violating privacy,” nor would that be a well-defined claim without a lot of additional nuance.
Instead, a forum of presumably technically savvy people are reduced to hysterics over implausible futures and a letter to ask Apple to roll back a change that is barely different from, and arguably better than, the status quo.
We need both - develop free software alternatives (which means to stop pretending the alternatives are good enough), and to get real about supporting legal and governance principles that would protect against abuses.
If people want to do something about this, these are the only protections.
icloud scans for those faces
finds those faces and ties them to other ID accounts via face - then via fingerprint recognition to a device, and to a location based on IMEI etc.
Apple's platform is literally the foundation for the most dystopian digital tool-set in history...
Once the government is able to crack the apple war chest, everything is fucked.
The "one in trillion" chance of false positives is Apple's invention. They haven't scanned trillions of photos and it's a guess. And you need multiple false positives, yet no one says how many, so it could be a low number. Either way, even with how small the chance of it being wrong is, the consequences for the individual are catastrophic. No one sane should accept that kind of risk/reward ratio.
"Oh, and one more thing, and we think you'll love it. You can back up your entire camera roll for just $10 a month and a really infinitesimally minuscule chance that you and your family will be completely disgraced in the public eye, and you'll get raped and murdered in prison for nothing."
So iCloud Photos circa 2020 [and Google Photos and Facebook and Dropbox and OneDrive] aren’t a risk you should be willing to take.
This feature doesn’t change anything in that regard; the scanning was already happening.
Apple's new policy is: "of course your phone is scanning and flagging your private files to our server - that's normal behavior! Don't worry about it".
Debian tries to be reproducible, so to avoid being caught they might need to control the mirror to so that they could send it to only me. I.e. if I'm lucky it would take a total of 2 people to put malicious binaries on my computer (1 with a signing key, 1 with access to the mirror I download things from).
It’s really disingenuous to suggest that open source isn’t dependent on trust, you just change who you are trusting. Even if the case is someone else is auditing that code, you’re trusting that person instead of the repository owners.
I’ll concede that at least that possibility to audit exists but personally I do have to trust to a certain extent that third parties aren’t trying to fuck me over.
With Apple you have to trust them and trust they don't get national security order.
I trust that if everyone who had the ability to audit OSS got a national security order it would leak and it would be impossible for many who live in other nations.
Suppose Debian's dev process happened at monthly in-person meetings where minutes were taken and a new snapshot of the OS (without any specific attribution) released.
If that were the case, I'd rankly speculate that Debian devs would have misrepresented what happened in the openssl debacle. A claim would have been made that some openssl dev was present and signed off on the change. That dev would have then made a counterclaim that regular procedure wasn't followed, to which another dev would claim it was the openssl representative's responsibility to call for a review of relevant changes in the breakout session of day three before the second vote for the fourth day's schedule of changes to be finalized.
Instead, there is a very public history of events that led up to the debacle that anyone can consult. That distinction is important-- it means that once trust is in question, anyone-- including me-- can pile on and view a museum of the debacle to determine exactly how awful Debian's policy was wrt security-related changes.
There is no such museum for proprietary software, and that is a big deal.
Most people don't care to follow the controversies of tech unless it becomes a tremendously big issue, but even then, as we've seen here, there are plenty of people that simply don't have the technical acumen to really do any meaningful analysis of what's being presented to them and are depending on others to form their opinion, whether that be a friend/family member or some tech pundit writing an article on a major news organization's website.
Trusting Apple presents a risk to consumers but I'd argue that for many consumers, this has been a reasonable risk to take to date. This recent announcement is changing that risk factor significantly, though in the end it may still end up being a worthwhile one for a lot of people. Open Source isn't the be all end all solution to this, as great as that'd be.
For regular Linux distribution there are maintainers updating packages from upstream source that can spot malicious changes slipped in upstream. And if maintainers in one district don't notice, it is likely some in onether distro will.
And there are LTS/enterprise distros where upstream changes take much longer to get in and the distro does not change much after release. Making it even less likely a sudden malicious change will get in unnoticed.
But, I suppose my iPhone was never really mine either. I knew that, I just never quite put two and two together...
So you are saying you have nothing to fear because you aren't hiding anything?
Is this a tacit admission that you don't want them scanning your phone for CSAM because they will find something?
I'm obviously not seriously accusing you of anything, just pointing out how your line of argument applies equally to privacy whether on the cloud or on your device.
No, I'm saying that if you're hiding something, maybe don't give it to someone else.
People here too often only think of these issues from the perspective of the type of person who browses HN. Apple is thinking of this from the perspective of an average iPhone users.
There is a huge separation in the importance that the tech community puts on the general concept of privacy while the average person rarely worries about it.
I prefer to think of it as being just one national security letter away from that happening.
Which is of course a schroedinger's cat kinda thing. It's already been sent. Or it hasn't. But why would the national security apparatus not take advantage of this obvious opportunity?
Anyone giving benefit of the doubt to this kind of stuff nowadays is IMO very naive or just poorly informed of recent digital history.
For a normal law enforcement context, say you bust 1 drug dealer, and are trying to find more. Maybe they have an image of a "contract" (document outlining terms of a drug deal) on their phone. You could find the other parties to that contract by searching everyone's phones for that image.
For a national security context, you could imagine equivalents to the above, you could also imagine searching for classified documents that are believed to have been leaked, or maybe searching for documents you stole from an adversary that you believe there spies are likely to be storing on their phones.
I'm saying documents here instead of images, but plenty of documents are just images, and I have little doubt that they could get this program to expand to include "scan PDFs, because you could make a CSAM into a pdf" (if it doesn't already?).
Then for version 2.0, they'll realize the INTERPOL, FBI, whatever can provide them a combo hash plus one string of text that can nail a known active producer. The holy grail. This small feature add to get so much closer to the original "moral goal" will prove too appealing to pass up. Now all the code is in place for the govt to pass over requests for data pinpointing mere common criminals.
Outside the US and other western countries, this could be a government (Russia? Chechnya?) forcing Apple to look for LGBT pride flags.
Just make it do equivalent of .* of all photos on device. It would be hard to argue, that's more difficult than scanning for specific hash.
And there is nothing specific about images. Extending this to scan arbitrary data is probably not that much code, depending on how its program maybe configurable.
In all these threads everyone is coming close to the crux of the issue, but I want to restate it in clearer terms:
There is a sacrosanct line between "public" and "private," "mine" and "yours." That line cannot be crossed by Western governments without a warrant. Cloud computing has deliberately blurred this line over time. This on-device scanning implementation blows right past that line.
Our tools before the computing revolution, and our devices after, become a part of us. Our proprioception extends to include them as part of "self." A personal device -- a tool that should be wholly owned and wholly dependable, like pen and paper -- that betrays its user, is a self that betrays itself.
And cloud computing are corporations that take your self away from you.
And that was always going to happen. And people have been fighting that idea in the shadows for a while.
This issue is just bringing it to the foreground in a most intense way. "Think of the children."
They are coming for your self. They will take it. There's not doubt. There never was. Your self is too valuable and you've just been squandering it anyway.
Eventually you become Apple. Unless you start resisting, but that's hard. Just slooooowly warm the water. This froggy is fine. Nice and warm. Just don't touch the bottom of the pot.
This is a self-delusion, I am afraid. The line has been crossed more than once, and it will be crossed again. UK and Australian governments are just two prime examples of waving terrorism and pedobear banners as a pretext to get invasive with each new legislation, and the Oz government already has a new legislation draft to make it a crime to refuse cooperation with law enforcing services when they request access to the encrypted content (think Signal messages). Also, refer to the Witness K case to see how cases that are unfavourable to the standing government completely bypass a «trustworthy» Western judicial system, including the Minister of Justice.
CSAM is guaranteed to be abused under whatever new pretext politicians can come up with, and we won't even know that Apple has been quietly subjugated to comply with it in those jurisdictions. There will be even less transparency and even more abuse of CSAM in «non-Western» countries. That is the actual worry.
Sir James Wilson Vincent Savile OBE KCSG (/ˈsævɪl/; 31 October 1926 – 29 October 2011) was an English DJ, television and radio personality who hosted BBC shows including Top of the Pops and Jim'll Fix It. He raised an estimated £40 million for charities and, during his lifetime, was widely praised for his personal qualities and as a fund-raiser. After his death, hundreds of allegations of sexual abuse were made against him, leading the police to conclude that Savile had been a predatory sex offender—possibly one of Britain's most prolific. There had been allegations during his lifetime, but they were dismissed and accusers ignored or disbelieved; Savile took legal action against some accusers.
Asking the police to put some investigative effort in rather than a dragnet sweep of every photo on every persons device is not too much to ask.
Importantly this should be the hard-capped precedent for every sort of crime. There's always a ton of other ways these people get caught without handing over preemptive access to every innocent persons phone. Same with terrorism, murder, etc.
I see push for CSAM as an alignment of powers in the form of "The Baptist and Bootlegger." The well meaning moral campaigners are a tool for folks who seek to surveil and control.
Even if the cops and private security along with TLAs are actively spying on people on an extremely regular basis, we're better off if officially they aren't supposed to be, if some court cases can get tossed for this, that they have to retreat sometimes, etc.
This is why having Apple overtly spying on the "private side" is still a big thing even with all the other revelations.
The fact that the line is there gives us a reason and a foothold to fight back when someone attempts crossing it.
if (willBeUploaded) {
scanPhoto();
}
if (true) {
scanPhoto();
}
Photos for your own use that you do not upload to Apple’s cloud are completely unaffected.
But why are people only objecting now when they were already doing on-device analysis?
CSAM detection is a complete red herring. It is less of a general purpose scanning mechanism than the search function that was already there.
If a bit can be flipped to make the CSAM detector go evil, surely it can be flipped to make photo search go evil, or spotlight start reporting files that match keywords for that matter.
There is nothing special about this CSAM detector except that it’s narrower and harder to repurpose than most of the rest of the system.
As your immediate parent has already said: there isn't any process by which an image of a hotdog will be sent to someone else without my knowledge.
Neither "photo search" nor "spotlight" report anything to third parties currently.
On the other hand, making the CSAM code scan your text documents would be pointlessly difficult given that the other mechanisms exist.
It has already been added. That's the point here. The phone already has the code to report its own owner. People are rightfully pissed about that.
Also, adding a one-liner of code is a lot more than a bit-flip. That's the addition of a feature, which is a major business decision. Code that already exists is a lot closer to "misfiring" than code that simply does not. Flipping a policy config, OTOH, could be "explained" away a lot easier. In fact, Big Tech does it all the time! (Remember Facebook's "Oh. Sorry. It was a bug."?)
----
Please stop this poor attempt at moving the goalposts by saying, "But, similar code could be added at any time in the future! Why're you guys complaining now?!"
It's also not about where the scanning is happening, or whether that's new or old. The old scanning is not a problem, precisely because it does no reporting. The new scanning IS a problem, precisely because it does the reporting.
The former is not a problem on its own. The latter is. That's the point you keep missing.
This whole ‘now that they have a scanner it’s easier’ reasoning doesn’t make sense. iOS already had numerous better matching algorithms if your goal is to do general spying.
It’s way more complex than that, and has been engineered to prevent exactly that scenario.
I recommend you actually check out some of Apple’s material on this.
The idea that it’s just a list or fingerprints or hashes that can easily be repurposed is simply wrong, but is the root of almost all of the complaints.
I feel like you may not have actually read the paper in depth. The paper clearly shows this is a generalised solution for detecting content that perceptually matches a database. The "what" is CSAM today, but nothing about the technicals require it to be CSAM.
To answer your specific response of safety voucher and visual derivative:
- The 'safety voucher' is an encrypted packet of information containing a part of the user's decryption keys, as well as a low resolution, grayscale version of the image (visual derivative). It is better described as "backdoor voucher".
- The 'visual derivative' is just the image but smaller and grayscale.
None of those two technologies have anything to do with CSAM. Apple's statement that they will only add CSAM to the database is a policy statement. Apple can literally change it overnight to detect copyrighted content, or Snowden documents, or whatever.
> Apple can literally change it overnight to detect copyrighted content, or Snowden documents, or whatever.
No, Apple doesn’t run the database. NCMEC does.
Also, in the US, the 4th Amendment protects against the abuse you are talking about:
https://www.economist.com/united-states/2021/08/12/a-38-year...
It can target a pride flag just as easily as CP.
Why do you think you know that? You didn't say. It seems like that is how you should see it since they are using your processing power on your device potentially against you.
But Apple have circumvented that with NeuralHash. There's no way for the user to verify that only CSAM is being detected - which could be partially accomplished by having iOS only accept hashes signed by multiple independent child protection organisations (with some outside of FVEY).
If they only scanned icloud I can choose to not use that.
If they scan on my iphone my choice would be to stop using that and get a gphone instead.
Then it only runs client side and alerts if there are matches to CSAM hashes above some threshold.
The other way is it runs against unencrypted images in iCloud and they'd know of any match.
The new method also paves the way for the iCloud photos to be encrypted (they're currently not).
The detailed specifics of this would be interesting to know. If I accidentally flip iCloud photos on and then turn it off within a few minutes, would that give the system license to scan all my photos once? What if my phone is in airplane mode and I turn iCloud photos on? Will the system scan photos and create vouchers offline for later upload?
Edit: On a related note, will turning on iCloud photos now come with a modal prompt and privacy warning so you're fully informed and can't do it "by accident"? Pardon my ignorance if this happens already. I've never turned mine on and am not about to start.
This is exactly how you should look at it. They turned your device against you.
There are plenty of over very real concerns with this. For example, say some activist intern at Apple decides they don't like your politics and hits the report button, next thing you know, the police show up to your door and are confiscating all your devices. Just one of many potential abuses.
But the fact that my iOS device can potentially report me to any authorities, for whatever reason, is crossing a line that makes it impossible to ever own an iPhone again. I bought my first one in 2007 so I'm not saying this lightly..
Does anybody know if this policy will extend to macOS too?
If anyone in my circle of family, friends, and social network asks my advice, the formerly easy answer "Get a Mac, get an iPhone; you won't regret it!" is probably going to be replaced with something more nuanced ("Buy Apple, but know what you're getting into; here's a few articles on privacy....").
If it does come to macOS it will be part of Photos.app, as that's the only way to interact with iCloud Photos. I would recommend you to avoid that app and cloud in general if you care about privacy.
[1] https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
- Communication safety in Messages
[…] This feature is coming in an update later this year to accounts set up as families in iCloud for iOS 15, iPadOS 15, and macOS Monterey.*
- CSAM detection
[…] To help address this, new technology in iOS and iPadOS will allow Apple to detect known CSAM images stored in iCloud Photos.
- Expanding guidance in Siri and Search
[…] These updates to Siri and Search are coming later this year in an update to iOS 15, iPadOS 15, watchOS 8, and macOS Monterey.
Not to be, uh, flippant. Because this feels like a rather obvious slippery slope that Google will be compelled to slide down as well.
Feels like the only way to avoid such a thing would be dumb phone + Linux laptop/desktop.
Plan to get one as my next phone, although it should be said that both of these offerings are still a bit rough around the edges currently, but getting there
I don't see how it could be in the same way, or at least it only could be on brand new ARM Macs right? The thing is that at least on Intel Macs Apple simply doesn't control the whole stack. I mean, obviously, while more effort it's perfectly possible to get x86 macOS running virtualized or directly on non-Apple hardware entirely. So any shenanigans they try to pull there can get subverted or outright patched out. Without the whole hardware trust chain and heavy lock down like they have on iOS this sort of effort, even if attempted, becomes less threatening. I guess we'll see what they go for though :(. They could certainly make it pretty nasty anyway so as much as anything it's more of a forlorn hope they'll mainly focus on low hanging fruit.
You have people arguing over the technical definition of privacy, or on devices, off devices terminology. It doesn't matter. The user doesn't understand any of it. And they just feel exactly what you wrote. An intrusion of their private space by a company that has been telling you how much they value your privacy for a better part of decade.
These sort of technical explanation used to be a Tech Companies / Google / Microsoft things. Absolutely make sense to nerds, but nothing much to a layman. Now Apple are doing the same. If you watch their Keynotes, you should notice the increase usage of technical jargons or numbers for performance over the past 5 years. Something that Steve Jobs's keynote used to keep it to minimal.
So really, the steady state is just us regular folks getting scanned. As the years go on, what is defined as CSAM will morph into "objectionable material", and will then include "misinformation" like today. They say they won't right now, but where will things be in a few years?
That assumes the hashes aren't recorded permanently.
Sounds like “mission accomplished” for Apple then
"If you give me six lines written by the hand of the most honest of men, I will find something in them which will hang him." said Cardinal Richelieu in the 16th century.
I'm pretty sure my phone and online accounts hold a lot more data than just six lines. How do I know there won't be accidents, or 'accidents'?
I think warantless search is a much stronger argument, though.
Why not? The argument seems perfectly reasonable to me.
It's definitely how you should look at it because it's right. Once a device is compromised in this way, there's no going back and the erosion of privacy and expansion of scope will become neverending. If a capability exists, it will just be too hard for spooks to keep their fingers out of the cookie jar.
iPhone runs software which is in full control of Apple. They take active measures to stop general public from auditing its behavior (i.e. no source code, no API specifications, etc).
In my opinion, such a device can never be considered as belonging to their users or "trusted" in any reasonable way. The reason is that its root functionality always remains in control of a different entity.
I believe that such a relationship can more accurately be described as a lease.
Instead of providing specifications and source code, which would help establish the fundamental trust in their products and their behavior, Apple seems to increase their reliance on the "trust me bro" promises, which their current user base has accepted.
So far.
"...You can't have a backdoor that's only for the good guys. That any back door is something that bad guys can exploit."
"No one should have to choose between privacy and security. We should be smart enough to do both."
"You're assuming they're all good guys. You're saying they're good, and it's okay for them to look. But that's not the reality of today."
"If someone can get into data, it's subject to great abuse".
"If there was a way to expose only bad people.. that would be a great thing. But this is not the world. .... It's in everyone's best interest that everybody is blacked out."
"You're making the assumption that the only way to security is to have a back door.....to be able to view this data. And I wouldn't be so quick to make that judgement."
No matter the mental gymnastics now, it's still a pretty drastic departure from what it used to be.
Prior to this when a government tapped Apple on the shoulder asking to scan for what they (the government) deem illicit material, Apple could have feasibly say "we cannot do this, the technology does not exist".
Now the box is open, the technology does exist, publicly and on the record - Now we are but a national security letter, or a court order with an attached super-injunction, away from our computers and phones ratting us out for thought crime, government critique and embarrassing the establishment. At first it's the photos, but what about messages and browsing history or the legion other things that people use their devices for, when do they follow?
CSAM is beyond abhorrent, morally reprehensible and absolutely should be stopped. This is not in question. And I have no reason to doubt that there is sincerity in their goal to reduce and eliminate it from Apple's perspective.
But they have root; this system and it's implementation is a fundamental betrayal of the implicit trust that their users have placed in them (however foolish it was, in hindsight). It just is not possible to regain that trust now that it is gone, when the level of control is so one sided and the systems involved are so opaque.
And not only that, but with a little footnote saying "* Features available in the U.S." - in other words, publicly and on the record: this can be switched on and off on per jurisdiction. If another jurisdiction comes and says "here's my list of hashes of illegal images", how can Apple say "sorry, can't do..."?
You are also right that it would take a lot for me to trust Apple again and they are unlikely to do the things required for that since transparency, openness are not typically how they operate.
Agreed. And the same holds for this subversion of user control of devices. It too is beyond abhorrent and morally reprehensible.
One stokes an intimate and personal fear or revulsion, something that any parent or anyone with empathy will feel.
The second allows for quiet enforcement of the subversion of the relationship the state has with it’s citizens.
Both are awful, but in their own incomparable way.
iPhones running current iOS are not rare at all. Much of our society carries and uses them.
They are not equivalent in scale.
Subverting every iPhone in the world (or country) to spy for the pigs is a much worse crime than all of the CSAM in the world (or country).
In fact, it is the more insidious danger of the two, since it can easily strip you from all your freedoms while not evoking as visceral a reaction as the first one.
It is out-of-step with everything they've been doing up to this point, and it makes me wonder who has something over the head of Apple that we aren't hearing about. The stated benefits of this tech are far outweighed by the potential for harm, in my view.
If Apple pushes forward on this, I want to hear a lot more from them, and I want it to be a continuing conversation that they bear until we all understand what's going on.
I think this announcement would have been well known to cause a major outrage so why say anything? I'm wondering if this is a cover up for something else. Get people talking about this in the news and getting outraged about it while something else goes unnoticed.
Unrelated to Apple, but that's one of the big problems with misinformation. You can't get ahead of it because it's all made up.
that's my thought, too.
The Apple vs FBI narrative was a coordinated marketing move (coordinated between the FBI and Apple) to push that "paragon of privacy and security" brand message. It's false.
Apple explicitly preserves a non-e2e backdoor in their cryptography for the FBI:
https://mobile.reuters.com/article/amp/idUSKBN1ZK1CT
It's not out of step at all. Apple actively cooperates with the FBI, both operationally and in product planning/design phases.
Hence, farce.
Can you imagine "the screeching voices of minorities with critical thinking"?
The big picture of hyperconnected future in which automated systems will decide your fate is in place and it's running well. It is not perfect. Yet. But it will be. Soon.
You will find a way to cope. As usual. As a collective. As a "exemplary citizen with nothing to hide".
"A blood black nothingness began to spin. Began to spin. Let's move on to system. System."
The system will grow. Powered by emotional reasoning created by the social engineers, paid well to package and sell "The New Faith" for the masses.
You are not consumers anymore. You are not the product anymore. You are the fuel. Your thoughts and dreams, emotions and work are the blood of the system. You will be consumed.
"Do you get pleasure out of being a part of the system? System. Have they created you to be a part of the system? System."
Yes. The pleasure of collective truism. In the name of the "common good" - the big, the bold and the brave brush, build to remove any form of suspicion or oversight.
"A blood black nothingness. A system of cells. Within cells interlinked. Interlinked"
Wake up, Neo...
First reference I read about adding other countries as a done deal, and especially about an incredibly opaque "small number of other groups" being involved as well.
Man, after all they've said about being obsessed with privacy they're really not doing themselves any favors here. What a tragedy.
It applies to private actors acting as agents of the government, it doesn't apply to private actors who for private reasons not directed by the government conduct searches and report suspicious results to the government (other property and privacy laws might, though.)
But the 4th Amendment would cover Apple being forced to modify their system to scan for anything else the Government has told them to.
if they WERE NOT acting as an agent then the 4th would not apply. I think they way they get around that is not do not report to the FBI but instead to NCMEC a "non-profit"
you do not want criminals to escape justice to do you???? /s
If a private citizen, on their own initiative, searches your house, finds contraband, and reports it to the government, they may be guilty of a variety of torts and crimes (both civil and criminal trespass, among others, are possibilities), there is no fourth amendment violation.
If a police officer asks them to do it, though, there is a different story.
No. The courts have already explicitly rejected the entity that NCMEC is a private entity. NCMEC can only handle child porn via special legislative permission and is 99% funded by the government.
The searches here are lawful because Apple searches out of their own free will and commercial interest and when there is something detected their employees search your private communications (which the EULA permits).
This is also why Apple must review the matches before reporting them-- if they just matched an NCMEC database and blindly forwarded then to NCMEC then it would be the NCMEC conducting the search and a warrant would be required.
it has literally been inverted from it original purpose
"In the exercise of his rights and freedoms, everyone shall be subject only to such limitations as are determined by law solely for the purpose of securing due recognition and respect for the rights and freedoms of others and of meeting the just requirements of morality, public order and the general welfare in a democratic society."
That...varies. The right to a speedy trial can be waived. The right against enslavement cannot. You can consent to a warrantless or otherwise unreasonable search given adequate specificity, but outside of a condition for release from otherwise-constitutional more severe deprivation of liberty (e.g., parole from prison) I don't think it can be generally waived in advance, only searches specifically and immediately consented to when being executed. But I don't have any particular cases in mind, and that understanding could be wrong. But “constitutional rights can be waived” is definitely way to broad to be a good useful guide for resolving specific isssues.
There are limitations, of course. If you live off-post in private housing, your commander has no legal authority to inspect it. They can only go through your stuff if you live in government-owned housing.
That's not a waiver, that's a space warrantless searches are reasonable under the fourth amendment and Congress’ Art. I, Sec. 8 powers with regard to the military, etc. Otherwise, when we had conscription, conscripts, who do not freely consent, would have been immune.
> Apple employees have flooded an Apple internal Slack channel with more than 800 messages on the plan announced a week ago, workers who asked not to be identified told Reuters. Many expressed worries that the feature could be exploited by repressive governments looking to find other material for censorship or arrests, according to workers who saw the days-long thread.
"A mail cover is a surveillance of an addressee's mail conducted by postal employees at the request of law enforcement officials. While not expressly permitted by federal statute, a mail cover is authorized by postal regulations in the interest of national security and crime prevention, and permits the recording of all information appearing on the outside cover of all classes of mail."
The wording of the CSAM law is that content should be scanned when uploaded. That condition "upon upload" triggers the 3rd party doctrine.
Apple has gone above and beyond here, not the actual US gov. So, the bill of rights doesn't apply to Apple's decision to scan content on device, just before it is uploaded.
However, if they do review the material and see child porn they're obligated to report it.
Apple is engaging in this activity of their own free will for sake of their own commercial gain and are not being incentivized or coerced by the government in any way.
As such, the warrantless search of the customer data is lawful by virtue of the customer's agreements with Apple.
What's the commercial gain?
Apple is basically saying to governments: see our walled garden? The more you allow us to wall it, the more we can help you oppress your citizens.
Honest question: how do you know this, for sure? Or is your comment supposed to be read as an allegation phrased as fact?
Is the pornography the problem, or is the the abuse of children?
$13/month is nothing to Apple, and I really regret it since photos in iCloud is really convenient, however it feels unethical to contribute to the Apple services system when it will inevitably be used to hurt people like dissidents and whistle blowers.
It is arrogant to blow off security and human rights experts outside of Apple as simply confused or mistaken, when there are sufficiently informed and imaginative people within Apple to understand the practical implications of preemptively investigating users. Such contempt for users is also a sign of worrisome complacency.
On the plus side, Apple has seemed to crowd out interest in third party OS user interfaces. Knowing that Apple believes you're a pedophile until they've run image analysis software on your stuff is one way to motivate people. Maybe 2021 will be the year of Linux on the desktop.
Anyway I hope that more users sacrifice the convenience of Apple services now that its hostility is apparent, and I hope that more people tell their political representatives that it is time for meaningful updates to anti-trust legislation. I know that I was complacent when I thought that having a choice, where at least one of the companies didn't place ads in the start menu, was a tenable state of the market.
They just lost a few thousands from me alone not upgrading all my iDevices for next model year. Savings go to privacy advocacy organizations.
I, quite simply, don't trust them.
This project is probably going to cost millions, they won't catch any pedophiles, they will only scare them so they don't use Apple devices.
Nobody asked them to do it.
This is none of their business.
It can only creates new vulnerabilities, privacy issues and general annoyance.
That by itself makes it look ok.
Until you consider… there are a couple counterarguments.
One, Apple has not actually enabled such private encryption with the keys out of reach to Apple for iCloud backups.
Two, that child protection in other countries will sometimes be defined in such repugnant terms that it will compromise Apple fully to scan for the hashes provided by “child protection” organizations in those countries.
“Child protection” is in quotes not because I think countries will get away with shoehorning, say, terrorist content hashes in as purported child pornography hashes. It’s in quotes because the concept of child protection can be so wildly bizarrely corrupted in some countries for religious or ideological reasons. Who decides what is off limits for children, from having gay parents, to having friends of the opposite sex, to having an unislamic head covering? Well, each random government, of course, with Apple as the enabler, and individual and human rights be damned.
So while the most charitably viewed end game may be good, they seem to be papering over the real impacts this could have.
https://en.wikipedia.org/wiki/EARN_IT_Act_of_2020
Big tech companies are under a lot of US govt pressure right now to crack down on CSAM, most especially Apple because of their very low number of reports compared to most other tech giants. I think Apple saw this as a way to ease some of that government pressure while not jeopardizing their ability to use end-to-end encryption, which something like the EARN IT Act could effectively make illegal by requiring a government backdoor for all encrypted cloud services that operate in the US.
Apple probably saw the on-device CSAM scanning as a small, widely-acceptable concession to make that could prevent much bigger crackdowns, but maybe didn't anticipate the level of blowback from people seeing the CSAM scanning itself as an unacceptable government backdoor on their own device.
Mission accomplished? They can argue that encrypted iPhones aren’t being abused by pedophiles, and they can argue that alternative App Stores will be avenues for illegal material.
Firstly, and most importantly, this kind of backdoor is the kind of thing that makes big corps prohibit the use/purchases of devices.
Secondly, it seems rife for abuse: dont like someone who uses an ios device, msg them some cp and destroy their entire life.
I think people can understand, Apple can't have certain content on their servers. People have a much harder time understanding that Apple needs to make sure you don't have certain content on your phone.
> Apple needs to make sure you don’t have certain content on your phone. Excuse me, what? What does Apple care what I have on my device?
Except they're legally forced to care. Thats why this is happening at all. They have a legal obligation to care, encryption be damned. They chose to preserve encryption instead of preserve trust.
IANAL and could be wrong.
(f) Protection of Privacy. —Nothing in this section shall be construed to require a provider to— (1) monitor any user, subscriber, or customer of that provider; (2) monitor the content of any communication of any person described in paragraph (1); or (3) affirmatively search, screen, or scan for facts or circumstances described in sections (a) and (b).
I wouldn't be too hopeful about this. They can just disable this feature if the device is enrolled in MDM. For enterprise devices, the company itself is responsible for what's on them. And depending on the company, they may already scan all content for a bunch of stuff (like exfiltrating trade secrets or whatever).
I see this a lot. First of all, if you even have CP in the first place that seems quite bad and that you've made yourself vulnerable. Messaging someone will also reveal yourself as the sender. But my bigger question of this hypothetical issue is that Facebook, Google, Dropbox, Microsoft, etc etc have been scanning for CP all of this time on their servers. So it's not like this is somehow a new proposition. Thus wouldn't this already be a big issue through the many, many more people that use all of these other company's cloud products?
Sending a link to 4chan /b would probably be enough. Or a hidden iFrame in a forum post.
Its very very easy to get images onto someones device without their consent, getting them to upload them to a 3rd party is completely different.
If this really was such an issue we would have hit it long ago. CSAM scanning is going occurring for photos that would be uploaded to iCloud.
An iFrame full of CP downloaded from 4chan that fills the browser cache with CP just by visiting a harmless site would not go anywhere near Google Photos, Facebook messenger, etc etc
It would trigger a scanner checking the browser cache for CP images.
Those are just the obvious ways, by "rife" I mean there is any number of ways to get CP onto someones device without them knowing, unless the device itself is being scanned nothing would achieved by this.
And if you dont think people are that messed up, check: https://en.wikipedia.org/wiki/Swatting
Back when I used WhatsApp I got a lot of nice photos from friends and family there.
WhatsApp doesn't (at least didn't) have a way to export them but I could find them on disk and I had an app that copied that folder to a cloud service. Problem is this folder contained all images from all chats.
Now I'm not very edgy so most of my pictures are completely non-problematic (in fact I don't know of any problematic ones), but once in a while someone will join one of your groups and post shock pr0n.
I've long since stopped backing up everything since I use Telegram now and it allows me to selectively backup the chats I want.
But I wanted to mention just one very simple way were doing something completely reasonable could result in an investigation of an innocent person.
I’m not asking if people do bad things, I’m saying over and over again this is getting coverage on Hn and people are pointing to this hypothetical issue — yet this hypothetical issue has been possible for years on many more devices.
So accidentally stick a w in your siri teen porn search and you'll be seeing https://www.apple.com/v/child-safety/a/images/guidance-img__...
If it was photos taken on the device there would be no existing hash for the image to match.
"Does this mean Apple is going to scan all the photos stored on my iPhone?
No. By design, this feature only applies to photos that the user chooses to upload to iCloud Photos, and even then Apple only learns about accounts that are storing collections of known CSAM images, and only the images that match to known CSAM. The system does not work for users who have iCloud Photos disabled. This feature does not work on your private iPhone pho- to library on the device."
If WhatsApp does this by default then, ho boy is that a terrible implementation.
Furthermore there is a lack of imagination here:
- work from a hacked account (I've seen gmail accounts that have been "shared" with an attacker for long enough to let the attacker having conversations with their bank, i.e. attacker used the account but stayed hidden until the bank called to verify details.)
- VPNs exist.
- Russia exist.
- nasty images can be embedded into details of innocent high resolution photos.
- very soon they'll have to scan pdfs, pdf attachments and zip files otherwise this is just a joke.
- at this time it becomes even easier to frame someone
- etc
Remember, the goal is probably not to get someone convicted, just to mess up their lives by getting business contacts, family and neighbors to wonder if they were really innocent or if they just got off the hook.
That said, the bigger problem with having a secret database of objects to scan for is that it is secret: anything can go into it, allowing all kinds of fishing expeditions.
I mean: if you have such a wonderful system it would be a shame not to use it to stop copyright infringement in the west, blasphemy (Christian or Atheist literature) in Muslim countries or lack of respect for the dear leader (Winnie Pooh memes) in China.
For this to work one would have to send multiple photos and convince the other person to save the photos to their library.
One makes them look bad and the other makes them lose money. Which do you think they will choose?
There's a reason why Apple 0 day exploits sell for way more on the black market than Android exploits.
Trust is hard to earn, easy to lose and even harder to regain. User trust is such a huge part of Apple's business and success, it's shocking to me they'd damage it so much and so fast.
A system that exists but has "protections" can and will be abused. A system that doesn't exist can't be. It's really that simple.
Here's the big question though: did Apple executives not realize there would be a backlash against this? Or did they just not care? Either is hard to fathom. Apple's board and shareholders should be asking some pretty tough questions of Tim Cook at this point.
The fire was spreading, and Apple went along to announce it publicly. It was too late to argue for reading their materials properly. In-device scanning was on everyone’s head.
Some have argued, that leak was even intentional to lower pressures on new regulations. Or it was just whistleblowing.
But there is a lot of evidence, that Apple is going closer to full E2EE with this PSI system, and original goal might have been announce everything in incoming iPhone event. That might have ended even into the positive light.
FWIW, every article I see is about iOS exploits being cheaper than android because there are so many of them. Those articles all seem to be from around 2019, though. It seems like before 2019 what you said was true. I'm sure sure what prices are like within the last two years.
The opposite is true.
They're both shit though. Use FOSS operating systems you can administrate or don't bother with smartphones.
If somehow I found myself in an authoritarian state where porn was illegal just on apple devices do you think I wouldn't just switch to a different OS?
This is a bullshit excuse and we've seen it countless times. It is manipulation 101, get the victim to agree to a small request first then ask more and more.
I'm glad I never used apple devices and I'm not caught in their ecosystem.
Here's their privacy page: https://www.apple.com/privacy/
First sentence:
"Privacy is a fundamental human right."
Apple has adopted this as a "core value". Their executives use this exact phrasing.
It is an extraordinary claim. They don't say, "We think privacy is important". They claim it as a fundamental human right (i.e. applying to everyone, unconditionally) and wrap themselves in the banner of it.
How anyone can square that with statement, that self-described "core value" with their recent on-device image checking announcement, I really have no idea.
It will hurt their sales. It will hurt Apple.
They will have to walk this back or face devaluation.
They’ve already done a ton of damage to their brand. It’s really hard to believe they would sacrifice their brand like this. This is Apple.
They made their reputation on privacy.
You are 100% right. You cannot square the two.
They publicly announced a backdoor this time. They can’t un-invent the capability. You can now prove that Apple is lying about its concern for privacy by citing their own website.
I even think that Apple would have faced less backpack over this CSAM stuff if they hadn't used the "privacy" shtick just couple months ago.
“Wouldn’t it be wonderful if the people of future would willingly carry a surveillance device that they depend on everyday to communicate to others and carry with them everywhere?”
We need to move towards OSS and OSH phones with extreme urgency. We also need to get more people involved in Ham Radio.
Im not a ham because of some romanticised dystopia avoidance. It’s just fun to make things that you can talk to people on :)
> nice list for their shit to be confiscated.
I am not sure if this would go well in the USA.
Imagine if the government suddenly decided that they would open and scan everyone's physical mail into a permanent database. Why should email be any different?
Huh. That’s a big problem. If the American public doesn’t support privacy, we won’t have it.
This is the most important part that seems to be missing in every denial that Apple won't work with governments to spy on various types of content. _It's not Apple's choice to make._ They have no control over whether they're forced to or not.
https://9to5mac.com/2019/01/05/apple-privacy-billboard-vegas...
Apple 2021: but we will scan your photos anyway
I don't understand something they are scanning "using a database of known CSAM image hashes"
This means new photos will not be in that database, so what is the point of scanning for old photos as it will not prevent new crimes!
Did they not see Fight Club where frames were added on videos? Are they going to hash each frame from videos as well? All 60 fps 4k videos? I’m sure that a lot of people saw that movie, and some of them might be pedophiles. It really looks like an half assed solution
Sure, scanning a personal device vs. data stored in the cloud feels different because you supposedly own the device. But you don't own the software that runs on it, so the distinction seems artificial to me.
More like do you want the bank coming into your house to look for drugs because you have your loan with them.
Personally, I strongly favor privacy. I avoid all cloud services that don't involve local encryption. I have an old model rooted android phone with two firewalls. All apps have internet access blocked by default. I stay as anonymous as possible on social media. And so on. Recently my group at work moved our repositories to Github Enterprise. It made me uncomfortable. I voiced my concerns but they fell on deaf ears.
Brilliant, and illustrates how far off we are. I'll be scaling my level of surveillance following your lead.
“yeah but there was that one time Apple thought she had kid porn on her phone.. she said it was a false positive, police investigated never prosecuted etc., but…better to not hire, etc..you know..just in case”
All these concerns and outrage don’t seem to matter to most people, and a few boycotts here for Apple, a few more boycotts there for Facebook, and some more for <name-another-for-profit-bad-company> aren’t making any difference to their bottomline.
To say I’m seriously disappointed and frustrated is putting it very mildly.
see also:
https://www.wired.com/2006/01/att-sued-over-nsa-eavesdroppin...
https://www.theverge.com/2013/7/17/4517480/nsa-spying-prism-...
https://www.courthousenews.com/companies-staying-mum-on-fbi-...
"Apple bravely scans your text messages. Users greeted by 'Messages noticed this website contains misleading content, are you sure you want to share it?' in iOS 16." -MacRumors (2022)
Like I wholeheartedly get where you're coming from, but I'm not sure what realistic alternatives look like.
Where do I self-host NextCloud?
you can use whichever flavor of Android you'd like - a lot of people like GrapheneOS
GP said:
>... non-cloud services that provide me with automatic backups of everything I do if something happens to my apartment...
And my first concern is that a laptop or desktop (or my NAS) won't fare well if an apartment is broken into and the devices are stolen, or they get severely damaged in a natural disaster. What might a good mitigation be for that that doesn't involve spinning up a VPS via DO, etc.?
occasional offsite backups of all of your data
That way here's what you need to protect:
- Your whole-disk encryption keys / passwords
- Your Sia key
- Your password for Duplicati backups
[1] https://duplicati.readthedocs.io/en/latest/05-storage-provid...
Even with services like MEGA that tout keywords like "encryption" and "privacy" without end, look at their Terms of Service and there will be a clause stating that they will still give all your data to any law enforcement agency that puts in a request, including encrypted files. And they have no choice but to do so, because of the very same issue that's forcing the personal privacy debate with Apple into the spotlight - hosting any kind of illegal data, which is for all intents and purposes synonymous with CSAM, makes your service criminally liable in dozens of different jurisdictions. A public cloud provider that allows arbitrary file uploads and has absolute privacy cannot exist because of the necessity to allow law enforcement to investigate reports of the illegal hosting of CSAM in dozens of different countries.
Any expectation of privacy is a facade when dealing with the cloud. Even beyond the issue of privacy, you're still at the whims of the cloud provider in the end. Fail to play by their rules, and you're done for.
Yes, it's the cloud, but they don't control the backup software (it's open source, not developed by them, using public APIs) or the encryption keys (protected by a passphrase client side, use a good one).
There may be less programmery equivalents of that, which have scheduling built into the backup tool and so on. Just recommending what I've used.
(This advice applies to general purpose computers, not sure if there's an equivalent for android or iOS)
As far as I understand it, the CSAM hash database is part of the OS, which Apple can update in any way they like, including to read your messages or surreptitiously compromise the encryption of your photos (and they can force your device to install this code via a security update). We trust them not to do these things (they already have a track record of resisting the creation of backdoors), so I’m not sure why they we don’t trust them to also use this capability only for CSAM.
Sure, it would be technically easier for them to add the hash of the tank man photo (an example of something an oppressive government might be interested in) to their database after something like this is implemented, but it’s also not very hard for them to add scanning-for-the-tank-man-photo to their OS as it currently exists. Indeed, if the database of hashes lives on your device it makes it easier for researchers to verify that politically sensitive content is not present in that database.
Apple thought they found a clever way to satisfy a government request of "Can you guarantee your storage is not used to house XYZ". Apple then continues to be able to advertise 'privacy' while retaining E2E encryption (or at least have E2E across all services eventually).
What they didn't forsee is the potential slippery slope backlash that the IT community has become concerned about.
The scanning feature could be used in more nefarious ways (if modified in future). For example. The hash checking could be altered to check and hash against each photo metadata field instead of the photo itself.
Now we can find who took photos in a particular location at a particular point in history while still retaining E2E!
Would it go that far? Or can we trust Apple to stop moving the line in the sand?
If so, I would love to hear what assures you so much about this.
A couple reasons I don't believe a single word:
* After years of the surveillance state begging and pleading about compromising encryption, they went silent after talks with Apple and Google, over about the very time it can reasonably be expected it would take to develop this "feature".
* The number of pedos this could potentially even catch is supposedly so small that it is a compromise of everyone's rights … an inherent contradiction of not only Constitutional laws, but the very concept of innocence before being proven guilty. Not to mention that we have long been told there are no pedo rings, even in spite of the fact that even without this "feature" the last administration broke up more of these pedo rings than ever before. Why do we need this "feature" now then?
* Any actual pedo rings will easily work around this feature simply by altering images and changing the hash in various ways too numerous to list right now.
* Innocent people could easily be swept up in this if they happen to have images that hash to something the feds are comparing against, within the threshold of tolerance. What happens then, a secret court provides a secret court order to penetrate your network and devices to confirm whether you are a pedo because you took pictures of your grandchildren in a bathing suit?
* oh, did I mention the mountain of history of nothing but abuse and lies and lies and abuse in every single way possible and even to the point that he people exposing the not just illegal acts, but evil acts, are deliberately targeted by the state?
Why anyone would believe Apple, let alone the government that is clearly suspiciously standing in the background, looking away, whistling into the sky trying to act as if it has nothing to do with this, is beyond me. It's all just lies, front to back, top to bottom, left to right and in every other dimension.
What this clearly will be misused for is to identify or fingerprint dissidents and worngthinkers of any kind, including those who thing they are now rightthinkers, who will find themselves on the other side of the divide when they've expended their usefulness.
Governments can compel companies to do stuff.
The only option for the truly privacy conscious is to not use a company provided OS. The future is in buying a phone and flashing the ROM you want.
Until even owning a non-standard phone becomes illegal.
A similar scenario was found in Stallman's disturbingly-prescient story 24 years ago:
https://www.patrick-breyer.de/en/posts/message-screening/?la...
The short version is: all communication providers may include back doors and analyse messages of users with the same excuse. The follow-up is to make this mandatory. In practice, this will be a ban on any end-to-end encrypted communications.
https://www.latimes.com/california/story/2021-05-05/lapd-off...
I'm going to start rallying for my employer to dump apple as a vendor if they stay the course.
The UK for example can throw people in prison if they don’t hand over their passwords/keys.
Admitting this however would by necessity lead to admitting that citizens of the US/EU have almost zero say in how their government legislate encryption. All these countries wanted to subvert encryption for decades already and they have all the time in the world, unlimited budgets and every other possible tool in their arsenal.
All this HN rebellion is a storm in a teacup. Some innocent dude will get in trouble and everyone will talk about how “tech companies need to be regulated” and then they’ll go back to “we should trust the government to police us” after a few weeks of burning through their outrage. Always exceptions. Always surprised. Always trusting.
Makes me wonder if the management at apple feels the current stock of CSAM content is getting stale? /s
At a technological level, this highlights the difference between perceptual hashing and image classification, which (ostensibly) can extrapolate from the training data. Regardless of how you feel about running this thing on your phone, the approach they chose has some significant perverse incentives.
>"I am not afraid of personally being found with child pornography, but I don't want Apple searching my files looking for criminal intent"
But I would argue that you SHOULD be afraid of personally being found with child pornography.
First and foremost, this feature could be exploited for harm. If anyone ever wants to hurt you, all they have to do is get access to your unlocked Apple device long enough to drop CASM into a backed-up folder, and you get reported to the feds.
Or, way more realistically, maybe you downloaded some amateur porno that you didn't know had underage participants in it, and you saved it where it got backed up to the cloud, and two years later the government has added the video to their known CSAM database and you get reported to the feds for having it.
Given how the mere accusation of child sex crimes is enough to destroy careers, marriages, and lives, I see absolutely no reason anyone should trust their Apple devices with anything should this become normal practice.
And if this becomes normal practice, it follows that CASM won't be the only thing they end up looking for.
1. I understand Apple doesn't like to discuss its plans publicly, but rather present the final product when it is ready to ship. But if anything was a good candidate for one of the rare exceptions, then this, no?
2. Why announce unambiguously that this feature is "available" in the US only at first, in other words, can be enabled/disabled/configured per region? What is the plan when country X comes and says "fine feature you've built there, here's our list of perceptual hashes of illegal imagery, and the contact data of our competent authority to notify in case of matches."? Replying "ah, sorry, we can't" is not going to fly anymore, ever (!).
Even if they walk this back, what defence will they have left in the future if any government requires them to implement this odious feature and scan all images on-device and notify some authority if anything untoward (in that jurisdiction) is found?
The next Apple iPhone upgrade is on hold, until all plans about CSAM scanning on-device is cancelled.
Privacy matters.
This could marginally reduce Apple's leverage against evil governments seeking to pry into iPhones since "if you can do it for CP, you can do it for tank man". But with Apple already being the dictator of all OS-level code running on your device, this capability isn't anything fundamentally new. Do we have any guarantee that certain devices in China haven't received a "special" version of some software update that introduces a backdoor?
Personally, I have an Android and hope that my next phone will run some flavor of Linux.
When my daughter was three years old, we started giving her some IPad time.
She learned about Kids YouTube, and wanted to become a “Youtuber.”
Unbeknownst to us, she began making extremely long, detailed videos teachings other kids how to perform her latest accomplishment: how to use the potty.
She then pushed a series of icons which, she believed, uploaded the videos to YouTube for the world to see.
She then offhandedly informed us that she had been teaching other kids how to use the potty. By YouTube.
Much screaming and shouting ensued. Ultimately we determine she wasn’t uploading anything, and couldn’t even if she knew how.
She did however have several hours worth of videos showing th world how to use the potty.
She was quite upset when we deleted them.
Leaving aside debate on the “scanning photos against a database of hashes”, I hope that the parent-enabled features are useful.
Is it possible to just scan photos locally for child pornography, encrypt those photos, and upload those encrypted photos and the 'yes/no' answer that the scan returns plus some cryptographic proof that the actual scan (some calculation) was performed on the uploaded encrypted photo?
If this was possible it would enable privacy and also help prevent child abuse (or at least prevent child pornography from being stored on iCloud).
This CSAM scanning will do effectively zero to stop children from being abused. It will only serve to breach the client side of E2E across the industry. Apple's privacy credibility is shot. We need to push hard for true open source mobile.
Today's mobile duopoly (including the look don't touch that is Android), is the digital yoke of our times.
Support the Calyx Institute: https://calyxinstitute.org/
That's what the CSAM initiatives sound like to me.
So, no matter the criticism, Apple won't seriously be pressured to reverse course, and after that the door is open on incremental increases in content monitoring.
(Not to mention the problems that could arise from false positives.)
Because if there's one thing the whole Jeffrey Epstein thing taught us, is that there's nothing more important to the authorities than children, and they'll do anything to protect them, no matter the cost, even if an actual prince or a current president is involved.
Oh wait, hang on. Actually nothing happened and they don't give a shit about kids.
So why do they want this again?
Personally speaking, Apple denying breaking into a phone in 2016 was indeed one of the reasons I believed them when they said they stand up for privacy. This is now developing a foul after taste - when it really counts, they don’t publicly stand up against governments.
I’m concerned that Apple’s stance on security has been compromised and it is time to dump the platform or find an acceptable modification.
I’m surprised there’s no gist being linked on HN about to get people bootstrapped for secure device backups as an alternative to iCloud (and without a jailbreak).
From my network logs, my iPhone is always connected to Apple servers.
Moving the locked iPhone a little bit on the table, very often it establishes a connection to Apple. Why?
If I block all Apple servers, my iPhone tries continously connect to Apple servers - which seems to drain the battery very fast.
Some years ago, I asked Apple wheter it is possilbe to switch of these 'connections'. Answer: No, I have to trust Apple.
WH
It's one thing to compute a simple hash. It's another entirely if they're using some type of locality sensitive hash since that very much has the possibility of false positives and would be a major invasion of privacy (which it already is).
What else are they going to do with all that cash they're sitting on?
Also in this way they are being patriotic and supporting 'American innovation'; i.e. the American Intellectual Property regime [1].
Seems unlikely, but stepping into "what if", what are people's thoughts on what it would like for Apple? Does it mean Cook resigning, firing a team. How do we get from here to there? Is CSAM on iPhone the next AirPower where it just gets cancelled?
For every "completely authoritarian" regime, you have handfuls of (historically democratic) governments on their way toward authoritarianism. In these countries, it's often not the laws calling the shots but the politicians that happen to be in charge at that moment.
Even in democracies with laws as guardrails, presidents will push for data on political opposition - at times threatening techies along the way (see: Brazil jailing WhatsApp execs for access to data [0], Twitter/India raids, or, in the USA: Trump DOJ asking Apple to turn over info on House Dems [1])
Simply "doing what you're told" around the world (no-questions-asked, aka "being in the arena" as Tim Cook says) will turn you into an enabler of authoritarianism, and not just in totally FUBAR countries like China but in many nations with a lot of freedom at stake.
0 - https://washingtonpost.com/world/national-security/senior-fa...
1 - https://www.techmeme.com/210610/p51#a210610p51
From: https://twitter.com/SpencerDailey/status/1426008754143776771
It's mind boggling to imagine that a company comes up with that stance and advertises it so proudly without ever having considered that it is at odds with what governments around the world will want.
"I know we said privacy is a fundamental human right but that was before the U.S. government asked us to give them your data"
What prevents CSAM from being compromised on the device by a rouge state actor to smear and false flag opponents ???.
You're not telling me they can access the clouds right? /s
Is this the beginning of the age of vigilante justice?
How is this much different?
Snooping around on users devices, goes against anything regarding privacy they seemed to stand for. It makes me sick to think this might even be legal. It is also sad they completely ignore any valid concerns about this intrusion.
Whats the next step if we as a society accept this behavior? Homepod listening in, until hashes of conversation snippets match some prehashed terms and sounds potentially related to child abuse and then alerting the authorities?
You've become Microsoft. I might as well buy a facebook phone.
Everyone who took Apple at their word was already doing extreme mental gymnastics because Apple's privacy stance was a farce on borrowed time to begin with. Now it's just blatantly obvious to everyone.
It's kind of funny how I am very suspicious of governments and politicians but when it comes to a trillion dollar corporation, I was giving them more of benefit of doubt. I was wrong.
Apple is a public corporation at the mercy of several superpowers. Economic incentives and the people with the guns make the rules. Full stop. Apple's privacy stance was always one desperate/amibitious business unit or bureaucrat away from complete collapse.
Considering how often Apple gives up customer data without a fight when requested to by the government, I don't really believe that.
According to Apple, they respond to government data requests with customers' data ~85% of the time, and 92% in cases of "emergencies"[1].
Apple gave customer data from over 31,000 users/accounts based on FISA requests and National Security Letter requests in the first half of 2020 alone[1].
During that same 6 month period, Apple provided customers' data to the government's data requests (not FISC related) about 9,000 times[1].
Doesn’t make Apple look any better though. I’m curious about how often Google and Facebook hand over information.
To clarify: The 9,000 figure is the number of individual data requests, but the requests themselves asked for data from a total of ~120,000 different users/accounts. Multiple users' data can be requested in a single request.
Say, what is Apple's advertising policy? Hmm. It looks very familiar, doesn't it?
This is merely a reminder that Apple was founded in and most HN readers live in the United States, a nation founded on the idea that, without checks and balances, any power that can be abused will be abused. Without something legally binding and someone holding Apple to their word, their privacy stance was never going to be anything more than a temporary ploy in their ultimate goal: to make more money.
Too bad. Scumbags they are.
Yes, they do a lot for privacy. But when it comes to their bottom line, they also have a record of compromising on privacy (and consumer rights in general) to preserve business with less than freedom-loving countries such as China, the UAE, Russia.
It is sometimes difficult to criticize them for this because the financial loss to them would be huge if, say, China kicked them out of their market (and utterly devastating if they kicked them out of manufacturing in China), and because people like to make the argument that iOS is (probably) "still the the bets option for privacy in China" - but it doesn't change the fact that in Apple's hierarchy of priorities privacy ranks lower than making money.
To give a concrete example: if Apple allowed sideloading of apps as Android does, Apple would no longer be in the position to remove VPN apps on behest of China - but at the cost of opening up app distribution outside their own store, which means no free rent-seeking income from that anymore. They'd now actually need to compete on providing the best store for developers, which is obviously going to be more work and cost for them. So, instead they choose the "lesser evil" of putting themselves in a position where they are the only thing that stands (or rather: drops dead lie a wet sack) between an authoritarian state and people trying to circumvent that state's surveillance.
It's a good thing Apple shows more Courage(TM) when it truly counts, for instance when it comes to ridding us all of that terrible scourge of human existence, the 3.5mm jack.
Except that they are making money on overpriced headsets and acquisitions like Beatz.
I guess they have always compromised on privacy to make more $$, whether it is the crummy protections for browsing on iOS, selling out to Google, or the nearly required bluetooth. Now, I am sure there are a lot more things to add to this list..
Two Final thoughts:
From a business perspective this is a serious impairment on Goodwill.
Loss of Privacy is going to negatively impact Apple services.
We detached this subthread from https://news.ycombinator.com/item?id=28163326.
It took a six hour rollercoaster ride before flags killed it - not even a fair shake for anyone to vouch despite no flagged children.
1. Your comment didn't add any information—it was just grandiose, inflammatory claims ("extreme mental gymnastics", "farce to begin with" and "blatantly obvious")
2. Replies like https://news.ycombinator.com/item?id=28163531 are to be expected to such comments. This is the way that discussion degrades. (That was the top reply to your comment before I downweighted it, so the effect was a lot more obvious before that.)
Ok, three issues:
3. This entire subthread is way more generic than the better parts of the discussion. That's to be expected from inflammatory comments that don't add information. Generic threads are much more predictable and much less interesting than specific ones: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor....
"Extreme metal gymnastics" was the only inflammatory claim and it was the same language used by the person I was replying to. "Farce" has no more charitable a synonym [1] and "to begin with" does not make it any more or less inflammatory. "Blatantly obvious" - really? It was. Plenty of replies agree based on the merits of the argument. This isn't a flamewar
> 2. Replies like https://news.ycombinator.com/item?id=28163531 are to be expected to such comments. This is the way that discussion degrades. (That was the top reply to your comment before I downweighted it, so the effect was a lot more obvious before that.)
People get defensive sometimes, so what? Take the charitable interpretation. (and this is a deflection - I was asking about my comment).
"Please stop insulting... Don't act superior..." - how in the world did that get flagged!? How does this level of genuine politeness degrade the discussion?
"No one thinks you're any smarter by saying "I told you so"." - ...really? By HN snark standards thats praise.
> 3. This entire subthread is way more generic than the better parts of the discussion. That's to be expected from inflammatory comments that don't add information. Generic threads are much more predictable and much less interesting than specific ones: ...
That's a self fulfilling judgement (see above).
Worst case I'm guilty of extreme snark. Call it a coping mechanism.
[1] https://www.thesaurus.com/browse/farce https://www.thesaurus.com/browse/facade
That's a really bad worst case, far below the line the guidelines draw. Would you mind reviewing them (https://news.ycombinator.com/newsguidelines.html) and taking the intended spirit of the site more to heart? We want thoughtful, curious conversation here, not snark and fulmination.
> People get defensive sometimes, so what?
The problem is that it evokes worse from others and leads to a degenerative spiral, ultimately to flamewars, and in the long run to the site burning itself to a crisp. Remember that this has traditionally been the fate of internet forums and HN was started as a conscious experiment in trying to avoid that (https://news.ycombinator.com/newswelcome.html). Scorched earth is not interesting (https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...).
It's possible to learn not to provoke this kind of thing, and that's what we're asking users here to do. Of course one can't predict specifically how others will react, but one can definitely play the odds. Since the odds are roughly knowable in advance, we want users to post comments with a positive expected value, so to speak: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...
Please give me the benefit of the doubt that after a decade on HN I have read the guidelines many times and reply to the meat of my comment instead of the knee-jerk rhetoric (aka joke) at the end.
*> The problem is that it evokes worse from others and leads to a degenerative spiral, ultimately to flamewars, and in the long run to the site burning itself to a crisp.
IMO it demonstrably didn't.
Take this example where Pelosi, one of the richest members of Congress has personal calls with Tim Cook where she also holds her most worth stock of 17%:
https://greenwald.substack.com/p/nancy-and-paul-pelosi-makin...
It wouldn't be the first time... but it shouldn't be our first assumption.
Instead, in litigation caused by this scanning the companies of testified that they conduct the scanning out of their own free will with no coercion or incentive of the government, simply because they don't want their brand being connected to the distribution of child porn.
Implicitly, they'd rather be associated with unaccountable mass surveillance-- with products that violate human rights, and with a push of a button could be used to enable genocide.
They aren't stupid, so you have to assume that they think that the latter is less of a hit on their bottom line than the former. Lets prove them wrong.
They probably don't want “safe haven for child porn” to be part of their brand distinction from other online firms.
I’d say most people have iCloud Photos enabled, so I’m trying to gauge whether that’s true of Google Photos too.
Google Photos also does CSAM scanning, I believe
96% users keep defaults[1].
[1] - My estimate
> I’d say most people have iCloud Photos enabled, so I’m trying to gauge whether that’s true of Google Photos too.
I'd say most people don't have CSAM eitherway.