If it learns from humans and produces results like humans would... What's the definition of "limitation" here?
If it learns from humans and produces results like humans would... What's the definition of "limitation" here?
No, humans can understand "A red cube, on top of a yellow cube, to the left of a green cube" and draw it.
Dall-E gives you nonsense -- look at the picture.
It fundamentally doesn't understand things that humans do. That is limitation.
Of course it might be great overall and do things that humans can't, but it still has a limitation.
Gary Marcus has been hammering this home at least since 2018, probably earlier. It's more of a memory/association/recall machine than a machine that understands, and understands logic in particular.
"seven red lines, all of them perpendicular, some in green ink, some transparent, in the shape of a kitten"
Then it will respond:
"FU, it cannot be drawn!"
GPT-3 can complete this prompt:
clarify this scene description:
A bowl of fruit containing no apples
With The bowl contains bananas, oranges, and grapes.
So the semantic modeling is certainly doable.The designer photograph one is complicated and I'm not surprised that DALL-E misunderstand it.
But you're cherry-picking for the apples. It didn't consistently give an picture of five apples when asked for five apples. When asked for a bowl of fruit with no apples, it gives a bowl of only apples. That's obviously strongly sub-human comprehension.
https://www.booooooom.com/2016/05/09/bicycles-built-based-on...
People draw bicycle-like shapes, that don't quite work - they miss important details, or connect things that shouldn't be connected.
In other words: humans mis-draw things because we don't always do a good job of visually apprehending the world; DALL-E mis-draws things because it doesn't really understand the problem statement.
(I think this would be apparent if participants were asked to draw a bicycle, followed by a bicycle without a wheel. The two drawings would be roughly as "good" in terms of quality, but the humans will correctly obey the prompt and omit one of the two wheels. I suspect DALL-E would struggle with that prompt.)
And note that even if it did (which is definitely architecturally possible given progress in retrieval and demos like WebGPT), it still wouldn't necessarily be 'copying', in the goalpost-moving excuse.
Facebook has a neat diffusion model which uses retrieval on a dataset of images for generation: https://arxiv.org/abs/2204.02849#facebook You can see the exemplars it uses: they both greatly improve the quality of generated samples, but also don't look that much like the generated sample.
Maybe a better comparison would be a trained artist, who likely wouldn't make such egregious mistakes.
A counterpoint to the maze - it depends what we expect. If you wanted a maze designed, it would be of course a bad example. But if you wanted one painted, it's... good? If it was in a gallery, it could easily be a commentary on life with limitations, dead ends and problems we can't solve.
The computer has no understanding of what it’s doing. And I don’t think you can argue that it’s interrupting a brief for a commission. Is it even producing at???
A human needs to be there at the beginning to either dream up a concept and then describe it in painful detail to the computer.
Or a human needs to be there at the end to sift through the results to pick out the ones that actually match some concept.
In trying to think of a useful application for this: if an artist can lock down the style of the output, and then pump in a script and further description to story board things or get near final output, that could be quite useful.