Hacker News Reader

Comment by hansword | Hacker News Reader

I have a really bad feeling about this technique.

Change your webcam output to 3 fps, mumble something about a bad connection, and Bob's your uncle.

But more fundamentally:

If I am deepfaking my appearance on video call, I have full control of my machine. I thus can (with negligible lag) just grab the screen content. Once I have that, it's really not too much work to write a filter projecting this screen onto the deepfake image.

In other words: as a 'security by obscurity' method this might have worked for a while, with this now being public knowledge, I expect deepfake software to defeat this within the year, as this really is not a computationally or architecturally demanding change. It is ultimately just a filter, corresponding to screen content, mirrored and super-blurred. That shouldn't have any discernible lag with a modern computer.

View on HN

The next 10 years of software dev will be defined by the arms race of fraud generation (deepfakes, GAN images, GPT-X bots) and detector bots. What will end up happening is the detectors will catch the bottom 99% of fraudulent content, leaving the top 1% free reign over the dataspace.

And then there's the impact of false positives. You could even get a situation like we have with some CAPTCHAs where the best fakes are "hyperreal" and actually detect as less suspicious than many real images.

I personally like that deepfakes exist. Anything can be real and anything can be a lie. Plausible deniability for all.

Indeed. And it forces everyone to think critically. No pictures, audio or video online are trustworthy by themselves.

Even in science, you can never be completely sure of anything. "Sure enough" for purposes of human life, perhaps.

But you can invalidate anything:

https://en.wikipedia.org/wiki/Karl_Popper

Potentially resulting in even more division and radicalisation. The human tendency is to judge evidence strongly subjected to priors. Does the evidence allow me to keep my beliefs, or does it force me to change them. Evidence of questionable authenticity will be believed if it reinforces the priors, and disbelieved if it contradicts them, thereby increasing the divergence of held beliefs.

If you want to find truth, and you understand the nature of reality and perception, it will make you want to seek out counterexamples to your beliefs.

I am thankful for anyone pointing out inconsistencies between my own statements and reality. It will let me learn more easily, and refine my beliefs.

Yeah, there's a difference though between these aspirations - which I happen to share - and what is a statistically observable human bias that ought to shape our expectations of what's realistically going to happen.

You are right. I apologize for unconsciously trying to divert the conversation.

Indeed, I now realize what you are saying is more people will get more material for their confirmation bias.

Lol, no need for apology here! You have given me no reason to doubt that you are exchanging viewpoints in good faith.

>I am thankful for anyone pointing out inconsistencies between my own statements and reality

You will find there are people that are not appreciative of that and are willing to go as far as ending your life if you go too far in pointing out their faults.

> And it forces everyone to think critically

Overly optimistic.

We're not there yet, unfortunately. There has to be large scale public scandal with unmistakable proof of deepfaking for the public to comprehend the reality we are in.

The usual problem is of course, whether that plausible deniability will result in everyone being given equal benefit of doubt, or if will be (ab)used for selective enforcement.

The model here is about using a challenge window with an illumination sequence specifically designed to change faster than can be incorporated into live deepfake models on the fly without creating telltale artifacts from improper approximation of the illumination effects on the subject. Correctly faking the effects of monitor illumination is a non trivial challenge for well-chosen illumination patterns.

I read the article.

I was proposing to skip the (computationally expensive) step of approximation of the illumination effects on the subject within the live deepfake model, and just do a (computationally cheap) filter over the webcam image to be sent out.

I was saying 'correctly faking' might not be necessary, so let's fake it incorrectly. Of course I haven't tested it, but from reading the article, this seems to be enough to fool their detection model.

If I misunderstood this, please correct me, I am eager to learn. But please don't just repeat the article back at me. Thank you.

> Correctly faking the effects of monitor illumination is a non trivial challenge for well-chosen illumination patterns.

Surely not.

If you have a face in a specific orientation and lighting the whole point of deep fakes is that it generates “style transfer” of the incoming image, including lighting.

Doing it in real time is a computational restriction that makes it expensive, not difficult.

Are you really suggesting that the deepfake models can’t accurately reproduce structured light and shadow patterns?

I find that very surprising, and not what the paper says (“ We verified that the deep fakes created by Avatar- ify (github.com/alievk/avatarify-python) do not incorporate the environmental lighting and are therefore easily identifiable because in the presence of our active illu- mination, their temporal facial hue is flatlined with a nearly zero correlation.” <— they didn’t even test it except on a system that doesn’t do lighting transfer).

…so, this feels a lot more like “well, if you’re doing a crappy off the shelf meme using deepfakelive it’s pretty obvious” rather than “detect a convincing DF”.

The approach probably doesn’t scale against more sophisticated models or opponents with adequate resources.

> If I am deepfaking my appearance on video call, I have full control of my machine. I thus can (with negligible lag) just grab the screen content. Once I have that, it's really not too much work to write a filter projecting this screen onto the deepfake image.

The article already addresses this, in its fourth paragraph:

> The theory behind the approach is that live deepfake systems cannot respond in time to the changes depicted in the on-screen graphic, increasing the ‘lag’ of the deepfake effect at certain parts of the color spectrum, revealing its presence.

Reply on news.ycombinator.com