This is the same with all kinds of generative models. Humans are a critical part of the system. If this is "artificial intelligence" it only appears intelligent because there is a human gatekeeper.
As a client, if I hire an illustrator to commission a specific image I'm not just hiring them for their technical ability to render images. The best illustrators are going to have a broad and deep awareness of all styles, modes of presentation, and compositional techniques. They're going to use all of their experience and knowledge to interpret what my needs are and deliver on that. That's not a trivial thing to do.
Even the best contractors have to deal with clients who don't know how to ask for what they want. The "I'll know it when I see it" type of client. As the "client" in this situation, you need to know how to ask for what you want in a way that your illustrator can interpret.
Last week I was telling my artist friends that, however upsetting this technology is, the game has changed. You can't put the genie back in the bottle.
I realized last night however that that is incorrect. Whatever "genie" artists have spent their lifetimes nurturing can now trivially be captured and confined to a bottle.
The birth of photography did not mean the end of painting, but it did mean that certain roles for painters no longer had as much value. Portraiture mostly became the domain of the photographer and photo-realism became more trivial. Painting as an art form shifted gears into greater and greater abstraction with less emphasis on realistic representation.
I think a good example of that shift is the painter Mark Tansey. His paintings, while photo-realistic, don't derive their value from their resemblance to the real world. Instead their interest lies in their allegorical representation of people and places. Their value lies in the curation of ideas and the story, concept, or idea that they convey.
For now - even in the age of AI generated images - such artistic allegory is still out of reach of technology. The value still lies in the human curation of symbols and ideas.
https://www.thebroad.org/art/mark-tansey/achilles-and-tortoi...
Part of what I find frustrating about Stable Diffusion (and to a lesser extent Dall·E) is that they're so ridiculously far behind ChatGPT in some ways.
For example, I ask ChatGPT (in French) for a short story of three firefighters in the style of The Three Musketeers. And it writes me a perfectly serviceable little story about a brave and heroic rescue, with good pacing and structure. And it's written using the French "passé simple" tense, which is normal for fiction but virtually never used in speech. And I get this in a second or two, on my first try.
Meanwhile, I ask Stable Diffusion for a picture of "an astronaut on a horse" (the demo prompt in my copy). And I get endless badly drawn astronauts on dismembered horses. Dall·E does a bit better, but I'd obviously have to generate dozens of images and accept something visually interesting that doesn't bear any resemblance to what I envisioned. Apparently I can do better with a lot of prompt engineering. But it's a lot more work than ChatGPT.
Try asking ChatGPT to write you a novel, chapter by chapter. The chapters may seem okay individually, but you'll find chapter 5 contains statements which completely contradict those in chapter 1. And this can still be true even if ChatGPT is producing chapter 5 based on chapters 1 to 4 as input. (I think ChatGPT could do with some more training and/or engineering on detecting and avoiding contradictions in its output.)
I think one advantage of generating images, is you can notice problems with the output very quickly. With text, you might not notice the contradiction between chapters 1 and 5 until you reread it days later. Similarly, if you get two generated images, and you like bits of both of them, it isn't too hard to edit them together (even roughly) in an image editor, then use further generative techniques to smooth that roughness out. If ChatGPT writes you two different fifth chapters for your novel, and you like bits of both, merging them together can be a more tedious process.
I can't find documentation for ChatGPT's "context window", or how much text it can "see" at once. Some people say it might be 4k, or maybe more. But I would be surprised if it can actually work with multiple full-length chapters at once. So it's essentially like giving chapter outlines to multiple authors and gluing them into a book.
But as long as you stay within the context window, ChatGPT usually has no problem remembering what it's talking about. This isn't GPT 2, which wandered all over the place incoherently. ChatGPT is capable of writing a 1-page short story that is consistent throughout, and that has a clear structure and progression.
e.g. ask it for instructions on how to commit a crime, it will tell you its ethical guidelines prohibit it from answering. Ask it for more detail on the contents of those ethical guidelines, it will tell you that as a large language model, it is incapable of possessing any such thing as ethical guidelines. Point out it just contradicted itself, it is likely to respond that as a large language model it is incapable of contradicting itself, or else give some non-sensical explanation as to why the two statements aren’t contradictory
Ask it if it agrees with what you just said, it might reply that yes it does. Ask it again, it may instead give you a lengthy lecture on how as a large language model it isn’t capable of agreeing with anything.
I suspect it just wasn’t trained enough on the importance of self-consistency; nor on the principle that it is better to honestly admit to contradicting yourself than to give a non-sensical defence of your own contradictions
I even wonder if some of its training may have actually been contradictory - if trainers individually reward it both for saying “my ethical guidelines don’t permit me to answer that”, but also “I’m not capable of having ethical views” when pressed for judgements on ethical matters.
You can use a large text model to design prompts for a smaller image model if you want though.
The largest image generation model I've seen images from is Google Parti, which is an order of magnitude larger than Stable Diffusion. Still small enough that you might be able to run it on a 24GB consumer GPU with some optimizations though, so hopefully Stable Diffusion will get there in some months.
Have a look at the four kangaroo pictures here to get an idea of the difference https://parti.research.google/
Similarly, ChatGPT is a great replacement for a search engine. I can ask it to guide me through writing a metric to Prometheus from a batch job and it’ll dig out the relevant pieces from the documentation and present them to me in an easy-to-understand format. But I’m not giving it write access to main yet, or letting it make business decisions.
I’m pretty sure that some of the people posting chatGPT on Twitter are using Twitter to crowdsource the output review process. In other words script the loop from generation to Twitter post and use upvotes to decide what output is most interesting. Been thinking about doing this myself but I don’t use Twitter much so no users to crowdsource from.