Seeing faces in things: A model and dataset for pareidolia
mhamilton.net
mhamilton.net
> We can even use the deep feature representation of our trained animal and pareidolia detector to compute animal-pareidolia doppelgangers
When we as humans see a face in a cookie or something we quite often spend some time gazing at it noting what it is about the the arrangement of features that triggers the pareidolia. I have wondered if that urge to contemplate the similarity difference is an instinctive response to gather more data for future accuracy.
I guess in a similar vein there would be merit in collecting a data set of all of those quirks that generative AI produces to provide a point of comparison. Notably when most image generators produce artifacts it's because the local image generation is accurate but at odds with the global image. So fingers are beside fingers accurately but there are too many, or a swirl near some hair becomes hair but doesn't actually connect to the top of the head.
There are some datasets which collect 'hard' examples. One I love for the title alone is "When does dough become a bagel? Analyzing the remaining mistakes on ImageNet" https://arxiv.org/abs/2205.04596#google , Vasudevan et al 2022. The errors can be pretty interesting: https://arxiv.org/pdf/2205.04596#page=18 You can often see why a model might make a mistake and that you would too. (Like that first one of a swing - I would think it's some sort of large "tripod" too because I have no idea if I've ever seen a swing like that.)