Moiré no more (2021) | Hacker News Reader

Moiré no more (2021)

newsletter.shifthappens.site

When you get Moiré from the sensor of a camera or scanner, then there is nothing much you can do.

But extreme Moiré effect from resizing is not really Moiré. It's aliasing. And it typically happens because the resizing happens in a non-linear colorspace and/or with a poor kernel. Unfortunately this mistake is quite common in many image processing software.

Aliasing IS a type of Moiré. Take an image, and impose it on a grid of dots - and you get the same afect as from naive resampling.

Technically it's not, but your description is, because of the words "grid of dots", which means there's white space between the dots (as dots are circular), a necessary requirement for the Moiré effect. We see this in image sensors as well, where all photosites have a tiny amount of space between them.

Image resizing isn't a grid of dots, though. At the data level, pixels don't have space between each pixel to cause Moiré effects. Instead, interpolation of the colors leads to patterns similar to, but distinct from, the Moiré effect.

The Moiré effect is "not really there" in that it's a perceptual effect: you can reproduce it with an SVG with the same dot pattern on two layers, and transforming one of them a little. A human observer will "see" patterns that aren't actually there, and as you zoom in the effect disappears (if you're now thinking "because it's on a monitor!", then no: it's not. The same is true for two physical transparencies with the same dot pattern placed slightly shifted on an old school projector, and then changing the zoom on the projector)

Antialiasing is a data property: it can result from lossy interpolating of pixels to a new grid, and is there no matter how much you zoom in because zooming in on a pixel doesn't change its color.

The confusion comes from talking about Moiré patterns on grid based displays. Those will "lock in" or even intensify the effect, but they don't cause the effect.

There's not much you can do to fix it in software, but you can fix it in the optics. You can add a Moiré filter to the camera, which is an optical low-pass filter, ensuring that the image that reaches the sensor does not have any spatial-frequency components above the Nyquist frequency of the sensor (ie, twice the pixel pitch).

I obtained a similar effect with The Gimp and the wavelet decomposition plugin[1] that splits the picture in various layers containing details at different granularity. You can then simply remove the "bad" ones. On portraits, you can remove freckles and wrinkles very easily this way, too.

[1]: https://docs.gimp.org/2.10/en/plug-in-wavelet-decompose.html

So I tried it on his image and here's how it looks with default settings:

https://demo.intellique.org/nextcloud/index.php/s/SsgWGczo6G...

Much less work :)

Done since the mid-1960's :) but, nevertheless, always an interesting read!

This is classical (as in pre-GPU AI/ML) Image Processing. The reference text book in the field, "Digital Image Processing" by Gonzalez et al, shows how to do it: like the pioneers at JPL's Image Processing Laboratory (est. 1965) denoised Mars pictures from the Mariner probes using Frequency Domain methods like the posted article uses - see an example below.

[https://pixelcraft.photo.blog/2021/09/29/the-early-days-of-i...]

Speaking of which, with the advent of AI/ML methods, how are Image Processing and Machine Vision courses taught today at Universities?

- Has Python or other free languages replaced Matlab yet?

- Is Gonzales' course structure still followed somehow?

It's so classical, it can be done without a digital computer at all, using fourier optics! The true fastest fourier transform, done at the speed of light.

https://en.wikipedia.org/wiki/Fourier_optics

At the focal plane of a lens, an image is converted into its spatial fourier components. By adding blocking elements at that focal plane (eg. an aperture, or a piece of glass with marker dots), unwanted frequency components can be cut out from the image.

Aha - very nice!

My background with Fourier optics is mostly theoretical and the applications numerical.

Any references (books or online) exist about physical applications, like the one you mention?

I wonder what would be the more systematic approach than manually painting over the fft image? That constellation of peaks in fft should be reasonably easy to recognize (semi-)automatically, then you'd need to figure out good mask for those.. is it a circular blob, or maybe diamond/star shape? And what size is best? Etc etc.

It has been a long time since I have done anything in this domain (20+ years), but if I understand correctly, the original article author is essentially filtering out high frequency signals by painting over the bright FFT spots, essentially creating a localized low-pass filter or maybe band-pass filter.

FFT essentially just transforms data into its frequency domain, so I assume the representation shown in the article is just that, brighter spots indicate higher frequency information (which would come from all the little spaced dots), so killing those essentially smooths the noise out of the data.

I imagine that the same effect could be automatically achieved by identifying which frequencies to filter out to get the desired effect.

I dont believe that by painting over the bright FFT spots you are necessarily removing the high frequencies. Instead, you are removing the strong frequencies from the image. The strong frequencies cause peaks in the FFT image. Its the repetitive nature of the moire effect that cause peaks in the FFT image.

Ahhh, so perhaps the position indicates the frequency and the brightness indicates the magnitude of the frequency? That makes sense - in that case the atricle's approach feels more like you said, a manually applied band pass or comb filter, i.e. it either removes or attenuates signals within certain frequency ranges.

Indeed, the frequencies get higher further away from the center of the FFT image with the center being the "DC" component of the signal. The brightness, as you note, correspond to the amplitude of a given frequency.

you can think of it as a polar plot where: radius = frequency, angle = angle (as in points along the horizontal direction in the FFT correspond to e.g. vertical stripes in the image), brightness = power

There is an imaginary part to the FFT that contains the phase-shifts needed to add everything back up into the original image. You may have noticed that the "FFT of the image" is symmetric, so something else should be needed to compute the inverse.

Another nice property of the FFT is that convolution in image space is multiplication in frequency space, so multiplying the FFT with an image of a Gaussian (or any other linear kernel such as an edge detector) is exactly equivalent to applying the filter on the original image.

I think I 'd start with simple generated images first, like a sine wave rendered with dots, to get some initial insights.

Theoretically, you're attempting to remove the artifacts that were added back in by the halftone printing process and recover the original image. If you knew the exact noise spectrum that the halftone print added into the image, you could invert it. For example with some scans of halftone prints of known images, you could reconstruct that noise spectrum and remove it as best as possible. This would preserve, to your best ability, whatever high-frequency data existed in the original image.

If we were sure that the halftone resolution was smaller than the relevant details of the photo, then we could just use a low-pass filter (a circular aperture), removing all the high-frequency components from the halftone print, but keeping the details of the photo. However in this case, the photo has details that are at the same scale as the halftone.

So the next best thing is to try to infer the halftone noise spectrum and subtract it, which is hard because it's mixed in with the photo, and we don't know which spectral peaks are from the photo and which are from the halftone process. What we can guess is that the legitimate high-frequency information in the photo should be pretty random, meaning it will be smeared out across the fourier spectrum, while the halftone print will be strongly periodic - concentrated into dots that are nearly symmetrically distributed around the origin (with some radial pattern that depends on the print process), away from the low-frequency center point.

Based on this we guess that any big dots that are symmetrically placed around the origin are probably more halftone noise than image signal, and any grey areas between them are probably more real image data than noise. Then we fit a noise filter to the dots with a regression, doing our best guess to invert the noise, bringing the whole high-frequency zone to approximately a uniform grey.

And then we hope that the picture isn't a photo of a herd of small spotted leopards.

> All the demoireing guides tell you to apply a sharpening filter after this to compensate, but that’s like reheating cold pizza. You will never get back to where you were before, no matter how hard you try

Actually the noise from the moire can fake detail that wasn't there in the first place. Put film grain or a raster on top of a blurry image and it will subjectively improve in quality.

"This is why nobody will allow you to wear stripes on television:"

This used to be cardinal rule in television broadcasting but it's much less a problem nowadays with the increased resolution/line rate of HD TV and video processing (filters) designed to eliminate it.

Nevertheless, whilst once discouraged, moiré was often a useful tool as camera and CCU (Camera Control Unit) operators would use it to focus an image—the more pronounced the moiré the sharper the camera's focus.

As bad as moiré is perceived a much greater cardinal 'sin' in both television broadcasting and photography is lateral inversion. This is where the image has been swapped in the horizontal direction (in some places it was a dismissible offence), as it's a complete distortion/misrepresentation of the image, which in some instances, may go undiscovered—there being no visual clues to indicate the problem.

Lateral inversion is very obvious when writing, street signs etc., appear backwards but for some reason it's very much less so when images of humans are involved. Despite the fact that laterally-inverted images put men's clothes on women and vice versa—as blouses, shitrts, coats and pants flies appear the wrong way around—few people seem to notice.

There's much evidence for this, one I often cite is that there are images from WWII on the US National Archive by the US Army Signal Corps that are still laterally inverted after 70-plus years that no one has bothered to correct (it's not a recent scanning error either as the Signal Corps logo (which is embedded within the photo) is not laterally inverted but the image content is).

BTW, laterally inverting a film image reduces the resolution as the negative or slide is no longer in the normal focus plane.

Past discussion: https://news.ycombinator.com/item?id=28663719

Relevant xkcd: https://xkcd.com/1814/

Ah, thanks. I did search before submitting but nothing came up at the time (must have been my connection).

Reposts are fine after a year or so! This is in the FAQ: https://news.ycombinator.com/newsfaq.html.

Thanks! Macroexpanded:

Moiré No More - https://news.ycombinator.com/item?id=28663719 - Sept 2021 (112 comments)

Moiré no more (How to de-moiré a halftoned photo) - https://news.ycombinator.com/item?id=28394884 - Sept 2021 (2 comments)

> occurring when two similar frequencies overlap and create strange, pulsating patterns

I just made the connection between moire and beats. Nice!

https://en.wikipedia.org/wiki/Beat_(acoustics)

Here's my favorite "alternative" explanation of moiré:

sin(a) * sin(b) = 0.5 * (cos(a - b) - cos(a + b))

You may remember this formula from high school. The arguments a and b are basically frequencies. When you multiply two frequencies a and b, you will get their difference cos(a - b) and their sum cos(a + b). If two frequencies are similar (e.g. 101Hz and 100Hz) you get one very low frequency (101Hz - 100Hz = 1Hz) and the other one will be much higher number (101Hz + 100Hz = 201Hz). It's hard to see that higher frequency 201Hz, you have to come really close, but it is easy to see that low frequency, that 1Hz would manifest as 1 dark blob across entire area.

Hang on then, if you're just notching out certain frequencies by painting over them in a 2D FFT could you do the same thing by treating the image like it was raster-scanned video and passing it through a notch filter at the Moiré frequency, or for that matter a comb filter? After all that's how you remove PAL subcarrier speckle from composite video.

A certain YouTube creator wears striped shirts on camera a lot, but I feel like he should probably know better. I wonder if it’s an intentional, subtle troll?

What's the point of your comment without naming him?

There's literally a button for this in Photoshop, and I'm sure every other tool for photos / scanning.

So all the people who insisted nothing could be done were just ignorant of what could be done?

A sage reminder to all of us that sometimes it's better to upvote & listen (or in this case heart/retweet) to see if a better response surfaces than to offer our two cents.

Reply on news.ycombinator.com