Hacker News Reader

This is somewhat true, but then again it can be argued that the current style is a synthesis of already existing artistic practices from the past (aka 'there is nothing new under the sun'). And if trained on enough data, NNs do have this interpolation/generalization capability to explore spaces not seen in the training set (but seen within the training space distribution). In other words, does the distribution of the training dataset span all styles humans might find appealing? Maybe, maybe not.

So I made this comment quickly, I didn't expand on the last bit, which is what I think is key to understanding my argument. Basically, the required piece that makes art a social process is feedback, both from other artists and consumers (the public, in the past, patrons), basically whether they like, whether they talk about, whether it stays in the social conscious both for good and bad reasons, etc. Feedback performs the culling. That part thus requires other people, and cannot be reproduced within a black box currently, because you'd otherwise need to model an entire civilization's reaction to art, which is a coupled system which feeds back onto itself. Modelling very large, complex systems is a challenge still, for both traditional modelling (I happen to be a computational scientist) and for AI. AI has picked off some bits but it struggles in other domains. I feel like at this point, arguing "AI will get better at modelling complex systems" is similar to people who say "weather modelling with get better eventually," people really underestimate the difficulty with rounding out the last 10% of the problem.

Anyway, assuming you cannot model "society's reaction to art" in a black box, I think the only way to help refine DALL-E or other future AI systems is to "put humans back into the loop." You already see AI "extrapolating" from the dataset, and more often than not (which you expect from mere statistics) the results are ugly mistakes that AI researchers are trying to "fix", some examples of which the article lampoons as a draw back of DALL-E itself. The thing is creating "ugly" and having those pieces be culled is what is done by the external social process in art I'm referring to already. In this way, DALL-E doesn't replace art or artists, it just becomes another actor in the space, being subject to the same feedback that other artists are. You haven't really replaced artists, you have just added another competitor, so to speak.

Anyway, I went on a monologue, but one last thing I will say, I really think people making this argument seriously underestimate the size of the codomain of "possible styles humans might find appealing," especially since that set changes over time (that is, over the centuries of human history). I really don't know how someone can look at all of human history and think that "nothing new under the sun" really limits the size of that set. Just because a set is finite doesn't mean its "small." Cardinality of sets concerns things mathematicians make up in their heads, not things in the real world that require finite time to consider, count, process, etc.

Humans get to choose which images to use and which to discard, which might be just enough feedback. That said there’s nothing to say progression of the world requires humans in the process, perhaps eventually it will be AI all the way down.

I deleted a section of the last reply (I seem to keep doing that) because it was a bit of a "toke of the bong" sort of take. Yes, you can imagine a situation where it's just "AI all the way down," that is, you recreate a community of artists which are only AI and no human draws anymore. The thing is I feel like that situation, the AI looks a lot less like the NNs which are just matrices today and it looks a lot more like actual AI which is closer to what someone could imagine are people. That is what I can imagine in the future, which is a community of artists still, just not of humans.

Regarding an "all knowing" NN sort of black box that literally models the whole social process itself, I don't think is possible due to chaos, that is, sensitivity to initial conditions, like the weather. Large social systems are probably as sensitivity to initial conditions too and cannot be a priori modelled completely. That's kind of what I'll still stake as impossible in my opinion.

Your statements seem contradictory: AI creates new things based on what they are trained on, artists create new things based on what they are trained on. Soon enough, the training and the process will pull AI into the ability level of humans, and this is a huge step in that direction.

I guess I'll believe it when an AI trained entirely on 19th century art manages to reinvent 20th century art based on that alone.

A human couldn't do that either. It literally took all of humanity 100 years to do that.

Okay, then just take, say, van Gogh, and see if you can get it to independently paint The Starry Night (approximately) without having ever seen van Gogh's paintings or anything influenced by him.

Or pick impressionism generally and show that an AI that has only ever seen antecedents to impressionist painters can come up with something similar. It's hard to imagine something like DALL-E being able to do that.

Van Gogh was influenced by myriad artists before him.

The Impressionists were influenced by myriad artists before them.

Of course an AI could eventually be trained to develop a whole new art style.

Someone already said it earlier... there's nothing new under the Sun. It's just re-interpretations of what already exists. The value in art of any kind - song, writing, visual - isn't in creating something dramatically new and novel for the sake of creating something dramatically new and novel, it's an iterative, generative process that retells the same theme in a somewhat new way.

These models are extensions of that process.

Dont forget there is also lots of thinking involved. Artists understand their context and work with that. Like they do things to piss people of or to contradict something. They are also painfuly aware of the fact that work cannt be too novel to become popular. Often times they “train” their audience to come to work they’ve done years ago. Artists are not only repeaters of image patterns like current AI is.

So unless the AI becomes concious then i dont think they will replicate this process. On they other hand they will be probably able to “brute force” novelty with mashing random things and help of humans to sort it out.

That seems to be the challenge: If you train an AI model on all of the art Van Gogh could have observed, but not any of Van Gogh's own images, would it come up with anything like "Starry Night?"

There's a gap between what comes before and what comes after, and it's not totally clear than an AI model trained on what comes before can cross that gap in a way that a human can. Maybe? I don't think it's an "of course" thing, though.

I think of it this way: an AI model trained on existing images has billions of images to draw upon, more than a human does. But a human has some subset of those billions of images and their lived experiences, too. That is, as I stand at my desk typing these words, I see out a window where the bushes in front of my house are overgrown and need trimmed back, but the blooming trees across the street are visible through the bushes and providing splashes of color in a tangle of leaves that look nice to me. The branches of the post oak in my front yard reach out toward the street, and are suffused with dark green while the trees across the street are more medium green and yellow. As I walk around the neighborhood later, I'll see much, much more sensory input than even billions of static images supply.

Artists in the past were often constrained by the choice of medium, and things like having to mix chemicals to produce new paint colors dictated the palettes with which they could paint. Those artificial restrictions affected things in ways that might or might not be clear to a trained AI model.

Perhaps you tag the year of each image in the model training corpus so that it could, in theory, have some sense of progression, but would it choose to consider that feature? If you then asked for a painting as of 1880, could it make enough sense of that to avoid colors and techniques developed after that time, or at least techniques that built on techniques developed after that time? It's possible the future will involve more carefully trained models rather than the "everything all at once" models the current generation of models is using, but... I'm not sure.

In any case, I think the poster to whom you're replying is suggesting that someone like Van Gogh isn't just creating "more like what already exists," but that sometimes they are introducing something entirely new, at least new to the art world, perhaps drawn from life, or a fever dream, or strong drink.

>Van Gogh was influenced by myriad artists before him.

> The Impressionists were influenced by myriad artists before them.

Yes. That's why I'm saying, if you can train an AI exclusively on the stuff that could have influenced van Gogh, and coax it into reinventing van Gogh's work, then I'll believe you've developed an AI that's capable of something like the sort of innovation that van Gogh was.

I'm skeptical that what van Gogh produced can be reduced down to a function of the paintings that preceded him, in the way DALL-E is just a big function of all the images it's been trained on. Humans are "trained" on a lot more than images.

But human artists have a physical existence, memories of the life they have lived, social interactions and relationships, hopes and dreams for the life they wish to live, personal fears and regrets. A human is not an empty vessel or a pure reflection of a dead corpus of captured data.

They have a story to tell.

I’m not saying that a machine cannot be contrived to fake a story, but even a fake story is just a derivation from a collection of real stories.

What purpose would a machine have in telling a story about a life it never experienced? At best it could pretend.

I think some of the things you mention are captured in writing the prompt and selecting the final image

Sure, but the animating story is then coming from a human being (the writer).

Still, I believe that there are dimensions of visual storytelling that go beyond translating someone else’s words into pixels.

This doesn’t hold water. Most if not all art is derivative at this point. There’s only so many things you can do on a 2D surface and there’s more than enough material for new derivations.