It's probably paranoia, but if I'm journaling my life, business ideas, thoughts on contacts - I just want it to be encrypted on my devices before it goes into the cloud. I want each device to sync and decrypt locally.
And yet Apple Photos does full image library search for your friends’ faces, while not having to have all photos on your device.
Two common ways to do this: (a) bring each piece of content to the client device just long enough to add it to an on-device index, (b) let the clients update and re-sync an encrypted global index as content is added locally.
As Apple puts it:
”One of the best things about Photos is how it protects your privacy. iOS and iPadOS are designed to take advantage of the powerful processor built into every iPhone and iPad. So when you search your photos, for instance, all the face recognition and scene and object detection are done completely on your device.”
With Photos, Apple scans and builds the index locally on device, per method (a). If you enable iCloud Photos, it will also sync the index of fingerprints to labels in (b) style so your names, favorites, and key photos match across devices.
Apple’s quite tickled at making this available to you as a dev:
”On-device intelligence: Apple uses machine learning to enhance your experience — and your privacy — by using on-device processing so other people don’t see your data. We’ve used it for on-device image and scene recognition in Photos, predictive text in keyboards, and more. For example, the A13 Bionic chip and the Neural Engine in iPhone can recognize patterns, make predictions, and learn from experience, similar to the way you do. So your device can create personalized experiences without having to analyze personal information on Apple servers. Developers can use our frameworks, such as Create ML and Core ML, to create powerful new app experiences that don’t require your data to leave your device. That means apps can analyze user sentiment, classify scenes, translate text, recognize handwriting, predict text, tag music, and more without putting your privacy at risk.”
Impossible was too strong. Very, very difficult is more accurate. Tagging faces is much easier than FTS as well, as photos don't change (or you assume they don't change in terms of "who is in this photo"), where as your notes might be changing quite frequently.
Also you're gonna have 1-5 faces per photo max, where as with FTS you're gonna need to store a lot more than that per "note".