Good question. At the very least, we have a far deeper understanding of physical reality. Humans would not unintentionally (e.g., for effect) produce images of people with three ears, or of a bikini-clad girl seated on a boat with her head and torso facing us, and also her butt somehow facing us and thighs/knees away... yet I've seen both of these in the last week (sorry, couldn't find the reference, it was a hilarious image, looked great for 2sec until you saw it)
I admit that it is possible (tho I think unlikely) that this is a difference in quantity, not in kind.
One reason to doubt this is that Stable Diffusion was trained on 2.3 billion images. This is a vastly larger library than any human has seen in their lifetime (considering that viewing 2.3 billion images at one per second would take 72.8 years). Yet even if you count every second of eyesight as 'training', children under 1/10 of that age, who have seen only 10% of those images would not make the same kinds of mistakes.
Plus, the neuron/synapse/neurotransmitter and brainstem/midbrain/cerebellum micro & macro-architectures are vastly different than the computer training models. So, I think we can be confident that something different is happening.
>>so is it abstracting or not? where's the line between that and a mere statistical mashup?
Good question. There is definitely something we might call, or that might resemble abstraction. It's definitely able to associate the cutout images of an astronaut in a spacesuit from the backgrounds. It can evidently assemble those from different angles.
But it certainly does not have the abstraction to understand even the correct relationship between the parts of a human. E.g., it seems to keep astronauts' parts in the right relationship, but not bikini-clad-girls' parts (because of the variety of positions in the dataset?). There's no understanding of kinesiology, anatomy, or anything else that an actual artist would have.
Could this be trained in? I expect so, but I think it would require multiple engines, not merely six orders of magnitude more training of the same type. Even if 10^6X more training eliminated these error types and even performed better than humans, I'm not sure it would be the same, just different and useful.
I'd want to see evidence that it was not merely cut-pasting components of images in useful ways, but generating it from an understanding of the sub-sub components: "the thigh bone connects to the hip bone, the hip can rotate this far but not that far, the center of mass is supported...+++" as an artist builds up their images. Good artists study anatomy. These "AI"s haven't a clue that it exists.
>>to me the "search engine" case where it reproduces a specific training image seems like a failure mode that's distinct from normal operation
Au contraire, it seems that this merely exposes the normal operation. Insufficient images of that person prevented it from abstracting the person components from the background, so it just returned the whole thing. IDK whether it would take a dozen, hundred, or thousand more images of the same person, to work properly. But, if they all had some object in the background (e.g., a lamp) that was the same, the "AI" would include it in their abstraction.
(but I could be wrong).