Hacker News Reader

Comment by threeseed | Hacker News Reader

This article is full of inaccuracies.

a) People have extracted a pre-release version from an older iOS phone. It is not the one that will be released and not the same one Apple uses server-side to verify the process.

b) Adversaries can not reasonably break this process by flooding it with bad data across a range of compromised phones. The client side version is there to prevent this as is the fact it would require jailbroken devices with iCloud logins.

c) If you can't trust Apple not to include non-CSAM hashes in the database then how can you trust them not to compromise the operating system itself ? It's illogical. Apple verifies hashes against two sources and two manual checks are required thus needing multiple failure points in the process.

View on HN

For b), I can't see how creating spurious icloud accounts and spoofing it will be hard at it's face. I have made dozens of icloud accounts personally for testing in the past. They may take steps to address this attack vector, it's not an unsolvable problem but it isn't solved by requiring a device or icloud account as far as I can tell.

For c), the problem isn't just trusting Apple, for whom the hashes are somewhat opaque, my understanding is that they are provided by a governmentally regulated third party.

For c) you still need to imagine a scenario where Apple is deliberately complicit, because of the human review step. If non-CSAM hashes are inserted into the database, the human reviewers need to know what non-CSAM they're looking for -- otherwise they'll see the e.g. picture of a leaked document, observe that it's not child porn, and flag it as a false positive.

How would they know it is a false positive and not a CSAM image concealed in a document image that NeuralHash was able to "see"? It will be impossible to tell from a lowres picture with certainty.

My understanding is that NeuralHash is only supposed to be looking at visible pixels. Granted, I'm taking their word on that one, but I've not seen any evidence indicating that it does look for stenographically concealed images.

Not necessarily steganography. For example if the image has drastically lowered contrast, the lowres image will look like a false positive, but it will not be.

Okay, but now you're positing a scenario which is entirely pointless. Seriously, what would be the point? If you know you need to conceal the images to put them into your photo library, you surely know it's a stupid idea to put them there in the first place. It's all but impossible to find a supply of CSAM material without being made acutely aware of their illegal status.

Here's what I don't get: who are these people importing CSAM into their camera roll? I for one have never felt the urge to import regular, legal porn into my camera roll. So why would anyone do that with stuff they know could land them in prison? Who the hell co-mingles their deepest darkest dirtiest secret amongst pictures of their family and last night’s dinner?

If someone wants to conceal their CSAM library, I'm sure there's probably dozens of apps in the App Store that can store photos securely behind an additional layer of encryption.

Constantly flagging my account to Apple is an attack of its own. I don't need Apple to be complicit, just make a mistake by erring on the side of caution when reporting.

I don't see how there's a "side of caution" when it comes to images classified as "A1" of prepubescent minors ("A") engaged in sex acts ("1").

What does it take to have 30 or more images in your photo library that have been all been manipulated to be a perceptual hash match to 30 or more A1-classified images, while also being able to pass human review ("erring on the side of caution") with respect to them being maybe fitting the A1 classification.

There's no ambiguity in prepubescent minors engaged in sex acts.

The content reviewer isn't going to see an image in enough detail to determine that it is obviously A1 material, so yes, the content reviewer will err on the side of caution.

You seem to be fairly certain about exactly how much detail is in the visual derivative contained within the safety token. Care to share your source?

> you still need to imagine a scenario where Apple is deliberately complicit

Which is very easy to imagine. All I have to do is imagine a national security letter.

Isn't that no greater risk than the status quo, though?

Apple already has all these photos uploaded to their servers and knows the keys as iCloud Photos isn't end-to-end encrypted, so if a government can coerce them into cooperating with running scans like that, this new system doesn't seem to make any difference.

(If the argument is that they can be coerced into changing the new system to scan non-uploaded photos and reporting them outside of the iCloud Photos upload process... that's a threat that applies just as much to every phone. It's an argument for not trusting any OS that you didn't compile from source yourself.)

That is to say, it takes this away from the threat the article is presenting as "governments could sneakily use Apple's system to find dissidents despite Apple's best intentions" and into "governments could use the law to make Apple do things".

> Isn't that no greater risk than the status quo, though?

I don't have an issue with Apple scanning the stuff they're keeping on their servers, in accordance with their ToS. Have at.

The problem here is that they're doing the scanning on your device.

A national security letter can do a lot, but it cannot supersede the United States Constitution. Any attempt to co-opt the on-device search mechanism would be an unequivocal violation of the 4th Amendment of the United States Constitution—it's a search of private property being compelled by the Government.

(As distinct from Apple voluntarily searching for CSAM, which will be part of the terms of service. And distinct from being compelled to search cloud servers, which is exempt from 4A under the "third party doctrine".)

If you receive an unconstitutional National Security Letter, you can't just ignore it. You must endure a lengthy, expensive, and stressful legal battle. And due to the nondisclosure requirement in the NSL, you also have to do this in secret, without going to the public for support. Or you can fold and comply.

In this hypothetical scenario, the NSL would have to go to Apple the corporation, not to any individual. There isn't any individual at Apple that could implement the demand. There probably isn't even a group of ten individuals at Apple that could do it without other employees finding out. And as smart as you think the US Government is, there's no way they could possibly know who those ten people were.

So the letter goes to Apple. They have ample time and resources to push back indefinitely. Demanding that Apple implement a Government dragnet across tens of millions of private devices is so far beyond unconstitutional that complying wouldn't even be fleetingly contemplated as an option. In fact, Apple is the sort of company that would move heaven and earth to ensure that this unconstitutional NSL becomes public. If nothing else, their defiance of it would be fantastic PR.

Therefore it would be a massively stupid-ass move for the Government to try. They would have zero prospect of a positive outcome and they know it.

> There isn't any individual at Apple that could implement the demand. There probably isn't even a group of ten individuals at Apple that could do it without other employees finding out.

Sure there is. A number of different executives could do that. The CEO, if nobody else. The government sends the letter to the company's legal department. It's on the company to deal with it appropriately.

There's no need to prevent employees who will actually do the work from finding out.

Apple, like most other such companies, have complied with NSLs before, and they will in the future. The consequences for failing to do so are too extreme for it to be otherwise.

I'm not arguing that these aren't real problems, but neither are unique problems of the the client side scanning solution. It's the same or maybe even worse with server side scanning.

I'd assume FB, Google & co have some solution to b), so Apple should be able to figure out something. For c), at least Apple takes extra precautions by requiring the photos to be in two separate database provided by different governments. (Maybe other cloud providers do that, too, I don't know).

Yeah I agree that B is likely a non issue, at least not an issue that affects the users directly. But C doesn't make me feel any better because the fundamental issue is where the scanning happens. If it happens off my device it cannot happen to photos I don't send off my device. I understand they have policy governing this, but that's not addressing the core conceptual problem of crossing the network boundary onto what I pretend is "my" device.

It is YOUR device.

But the second you enable iCloud Photos sync, Apple has the right to not allow CSAM on THEIR servers.

Putting some extra hashes into a database - especially one tucked away in a corner of the product - is a lot easier than compromising kernel code or a driver. And in this case the database is not exclusively the work of Apple, so third parties have the ability to insert hashes.

Reply on news.ycombinator.com