This problem is perfect to exploit capabilities of modern CPUs, and neither of the projects does so. At least this one iterates over the y coordinates in the outer loop.
Nothing about what either tool does is noteworthy.
Here's some Python code (using OpenCV & numpy) that is roughly equivalent to OP's imgdiff. It is single-threaded but it'll be vectorised and it's 3x faster on my 5-year-old 2-core laptop (even counting the Python overhead):
import sys, cv2, numpy
f1 = cv2.imread(sys.argv[1])
f1 = cv2.cvtColor(f1, cv2.COLOR_BGR2GRAY)
f2 = cv2.imread(sys.argv[2])
f2 = cv2.cvtColor(f2, cv2.COLOR_BGR2GRAY)
threshold = 0.1
absdiff = cv2.absdiff(f1, f2)
_, thresholded = cv2.threshold(
absdiff, int(threshold * 255),
255, cv2.THRESH_BINARY)
cv2.imwrite("output.png", thresholded)
print("Different pixels: %s" % numpy.count_nonzero(thresholded))
$ /usr/bin/time python test.py water-4k.png water-4k-2.png
Different pixels: 89153
1.39user 0.34system 0:01.33elapsed 129%CPU
$ /usr/bin/time ./imgdiff water-4k.png water-4k-2.png output.png
Failure! Images are different.
Different pixels: 89142
Command exited with non-zero status 1
8.85user 0.10system 0:04.02elapsed 222%CPU
This C code isn't doing exactly the same thing; here it's calculating a global similarity measure by calculating sum-of-squared-differences and comparing the end sum against the specified threshold, instead of comparing each pixel-wise diff against the threshold.
It is straightforward C code but it's vectorised by the compiler: https://godbolt.org/z/-3uO1z
Excluding the overhead of reading & decoding the PNGs, it takes 80ms for the same 8400x4725 images as my parent comment.
Benchmark:
$ git clone git@github.com:stb-tester/stb-tester.git
$ cd stb-tester
$ make
$ ipython
>>> import _stbt.sqdiff, cv2, timeit
>>> f1 = cv2.imread("water-4k.png")
>>> f2 = cv2.imread("water-4k-2.png")
>>> timeit.timeit(lambda: _stbt.sqdiff._sqdiff_c(f1, f2), number=1)
0.081
In my experience, the only drawback of the Python API is the fact that it cannot utilize the multithreading module (probably does not release the GIL for long calls).
Good point about OpenCV's in-place operations. Sometimes it's tricky/impossible to do that in numpy if you need to implement something that OpenCV doesn't provide. For example the C code that I linked in my previous comment, we wrote as an optimization of OpenCV's `matchTemplate` when the inputs meet a specific condition (that both input images are the same size). In C we do the multiplications as we iterate over the images and we maintain a rolling sum in a single variable. In numpy you can't really do this, you have to multiply the whole array and then sum the result.
For 720p images our C implementation[1] was 100x faster than our numpy implementation[2], and 10x faster than numba[3].
[1]: https://github.com/stb-tester/stb-tester/blob/v32/_stbt/sqdi...
[2]: https://github.com/stb-tester/stb-tester/pull/566/files#diff...
[3]: https://github.com/stb-tester/stb-tester/pull/566/files#diff...
> I believe many (most?) opencv & numpy operations release the GIL.
Any idea how I can determine this? I am prototyping a real time machine vision application targeting 2x720p@240fps and I want to avoid writing any C++ for as long as possible.
(see https://docs.python.org/3/c-api/init.html#releasing-the-gil-... in the Python C API manual).
That's only called from the ERRWRAP2 macro here: https://github.com/opencv/opencv/blob/4.5.0/modules/python/s...
That macro, in turn, is called from gen_template_func_body in gen2.py: https://github.com/opencv/opencv/blob/4.5.0/modules/python/s...
And that seems to be a code generator that is generating the bindings for all the OpenCV functions: https://github.com/opencv/opencv/blob/4.5.0/modules/python/s...
As to testing this, on Linux I'd run `atop` to see if your program is using all available CPUs.