the entire "more data and pray" approach is how you's all end up with these problems. You want to see another stable diffusion generation failure, try asking Dalle or Craiyon for playing cards. In particular the back sides. Very similar to what you pointed out. Lots of repetition in the data sets, but non-exact repetitions. Different companies put different patterns and different stylistic choices on the same recognizable standard deck. The output the machine produces is frankly psychedelic.
Here's what you missed. The generation problems all happen under a certain length scale. Its things under a certain band of fine detail where the distortion happens. Normally you won't notice it, because normally there isn't anything but noise in that band anyway. Shove a SD generated image through a fourier transform to see what I mean.
And here's my very informed conjecture why it happens. The hint is right there in the name. Stable Diffusion. The generator network trains to de-noise images. The adversary adds Gaussian noise. That's a perfectly reasonable noise method, but it comes with its own spread and distribution in frequency space.
A very similar problem exists with dithering processes. One way to use a b/w screen for grayscale is to treat the gray scale values as coin flip probabilities. This is known as random dithering. Its simple, its obvious, it should work, and it does work surprisingly well most of the time. But it runs into the exact same distribution problem. When there are closely spaced stripes in the image you want to show, that stripe pattern gets completely washed out by the noise being added to fake brightness levels. Put another way, overlaying TV static on any image makes it impossible to see blobs of the same size as might happen by random chance. The generator network can't see certain patterns, because the length scale of those patterns are coincident with the length scale of the noise being added by the adversary.
This problem will never be solved by substituting data for comprehension. That's a one shot lesson worth generalizing.