Except if the watermark is terribly designed, that will not work. There is a lot of information that you can hide from the eye in a video.

Wouldn't you need three? Or, better still, do a bitwise 3 of 5 / integer median on the pixel values.

What would that accomplish? If you want to exclude differences, use AND.