Hacker News Reader

It remains to be seen if this is possible at all… let alone “easily” done.

Have any other big tech companies that scan for illegal images been shown to be vulnerable to this?

For sensitive content you can store like nudity, you can use some neural networks trained on a dataset you can manually check and sanitize, and you won't be susceptible to this attack.

But if you can't store the data and you are only allowed to store hashes, you become dependent of the quality of the hash database which is troublesome to verify, and therefore susceptible to poisoning.

How hard the game will be for the various players is not really the point, given that it's a game that shouldn't be played : only Apple know the rules and can verify them, and not even Apple should have legal access to offensive raw database to check that it hasn't be poisoned.

If they use some neural networks, it can easily turn into a PR disaster, once someone take their weights and use a Generative neural network to hallucinate offensive content based on the weight they release.

Apple’s documentation discusses how they use a neural network to generate the bits which get turned into the hash.

Reply on news.ycombinator.com