Meta's Un-Stable Signature | Hacker News Reader

Meta's Un-Stable Signature

hackerfactor.com

146 pointsementally25 comments

Related to this, the EU AI Act requires mandatory watermarking that is cannot be removed or is illegal to remove.

https://digital-strategy.ec.europa.eu/en/policies/eu-icons-l...

If Facebook already embeds user IDs in images (AI or no AI) I can only drool to think what kind tracking, advertising and mass surveillance opportunities are coming.

How can a watermark be unremovable?

The actual rules don't say that I think, it's more about the intention that the watermark is embedded with the image/multimedia itself, so it's persisting even if someone "right-click > save" the image or takes a screenshot, not literally regulated the watermark has to be unremovable.

> (Summary) The icon should be directly embedded into the deep fake or published text (except for creative works), unless equivalent alternatives are available such as a user interface overlay. The icon must be visible when content is reshared or downloaded.

It says the icons are optional. So that icon must not be what the other person was talking about.

> It says the icons are optional. So that icon must not be what the other person was talking about.

What "watermark" are they talking about if not the label/icons? The label/icon in question are what the whole "EU Icons for labelling AI-generated content" thing is about, someone correct me if I'm having a big brain fart.

I imagine the goal is for everything to use something like Google's synthid.

That sounds like one possible implementation, not the goal per se. The goal (the explicit/stated one at least) is to give people a heads up what's AI generated vs not, when that's unclear.

I don't see anything about watermarking in the linked article, it's about labelling requirements. It describes situations where you are required to disclose if an image was AI-generated.

A watermark is not just “transparency.” It can reveal what tool someone used, how they work, or that an image came from a stigmatized platform. In sensitive contexts—politics, sexuality, medical issues, protest material, or private expression—that can become surveillance.

I am working on Saigon Watermarks: https://apps.apple.com/us/app/saigon-watermark/id6777061197 for detecting and removing provenence markers in AI.

The tool also removes c2pa markers, which google is now linking the device that took the photo with the photo.

scary stuff.

https://security.googleblog.com/2025/09/pixel-android-truste...

No android version?

Still working on it...

I'm waiting on Apple to approve the MacOS version. After I will either focus on removing SynthID (currently not supported) or releasing android.

Another reason to drop both iOS and Android.

In favour of what, might I ask?

Keep in mind that the replacement must be a first class citizen in the society I live in. That means being able to easily transfer money using the app everyone uses where I live. It means public transport ticket apps. It means banking apps. It means iMessage, Facebook Messenger, Snapchat and Signal.

When generative AI can create such good fake images a valid c2pa linked to the source camera will become mandatory for an image to be considered authentic.

This is what REALLY pains me about this discussion: I am 100% about personal digital freedom, but I am also 100% opposed to promoting political violence and promoting theft and grift using generative AI. If C2PA is going to work towards one goal by being diametrically opposed to the other then it _cannot_ be a useful tool and we need an _actual_ solution. I was extremely excited by C2PA until today and now am only disappointed that there isn't already some better solution.

Edit: Thinking through this a bit more, I think the goal of _authenticating_ a photo using C2PA is still useful. If the goal is to remove them to get a "naked" image, that's fine, such an image is then inherently no more or less trustworthy than any other image. If the goal is to figure out how to reproduce a valid provenance chain on top of an altered image then I have problems with that.

This is a great statistical analysis and it was a pleasure to read, but I wasn't expecting the claims to be so poorly supported. There's also a reply from one of the Meta authors there, worth checking out.

Which claims do you think are poorly supported? I'm the author; I tried to include everything other people need to repeat the same experiments. I've even had two people write in directly to me, stating that they have been able to replicate my findings.

Or are you referring to the claims from Meta, Google, and Adobe -- which failed to hold up under independent evaluation.

You are also correct that one of the meta authors wrote in a comment. However, he demonstrated a clear lack of understanding regarding what makes the bits "independent" or how to resolve the independence problem.

> Or are you referring to the claims from Meta, Google, and Adobe -- which failed to hold up under independent evaluation.

This.

> However, he demonstrated a clear lack of understanding regarding what makes the bits "independent" or how to resolve the independence problem.

Yes, I didn’t want to call it out explicitly, but this is exactly the kind of thing that would have made my undergraduate statistics professor lose patience.

The point is to embed a particular signature, which was generated randomly, in the image. The distance function discussed is the same as the popcount of the xor. It's well know that the xor of random data with correlated data is statistically random. Hence, however well correlated the signatures of unwatermarked images may be with each other, they would show no correlation with the signature of a watermarked image. That is, unless the watermark by extremely bad luck happened to be near one of these clusters the author discovered. This does represent a genuine flaw, but an extremely minor one, and one that can be easily mitigated with no changes to the underlying algorithm,

How common is it for peer reviewed papers like this to be so far off their claimed findings?

“According to Google's peer-reviewed and published paper, they claim to have a true positive rate (TPR) above 99.97% -- meaning that they will miss their own watermarks less than 1 in 10,000 times. However, my own empirical testing found that is it much closer to 1 in 20.”

If there were bounties for invalidating peer reviewed research, I suspect this would be a lot leas common.

also, easily bypassed now: https://twotensors.ai/

There is actually an older method for countering steganography and adversarial image generation attacks: https://en.wikipedia.org/wiki/Gaussian_blur

AFAIK these methods claim to be blur resistant

Yes. These methods don't work reliably. Apply a blur and they still don't work reliably.

Reply on news.ycombinator.com