The chance of you getting enough false positives to reach the flagging threshold is one in a trillion. And then the manual review would catch the error easily. The FBI won’t be called on you.
What if it really was a very private photo?
As well this statement relies on the fact that no malicious actor out there is trying to thwart the system. The hashes used are deliberately chosen to easily produce collisions otherwise the system won't work. This almost certainly will be abused.
https://en.m.wikipedia.org/wiki/PhotoDNA
What Apple announced is a way to do it on the client device instead of the server. That has some security implications, but they’re more specific than just “hashes might collide.”
The problem is that the space for false positives is huge when you use perceptual hashing, and it gets even larger when you start looking for derivative images, which they have to do otherwise criminals would just crop or shift color channels in order to bypass filters.
How do we know this? It's not obvious to me that this system would work well for the edge cases that it seems intended for.