Try asking ChatGPT to write you a novel, chapter by chapter. The chapters may seem okay individually, but you'll find chapter 5 contains statements which completely contradict those in chapter 1. And this can still be true even if ChatGPT is producing chapter 5 based on chapters 1 to 4 as input. (I think ChatGPT could do with some more training and/or engineering on detecting and avoiding contradictions in its output.)
I think one advantage of generating images, is you can notice problems with the output very quickly. With text, you might not notice the contradiction between chapters 1 and 5 until you reread it days later. Similarly, if you get two generated images, and you like bits of both of them, it isn't too hard to edit them together (even roughly) in an image editor, then use further generative techniques to smooth that roughness out. If ChatGPT writes you two different fifth chapters for your novel, and you like bits of both, merging them together can be a more tedious process.
I can't find documentation for ChatGPT's "context window", or how much text it can "see" at once. Some people say it might be 4k, or maybe more. But I would be surprised if it can actually work with multiple full-length chapters at once. So it's essentially like giving chapter outlines to multiple authors and gluing them into a book.
But as long as you stay within the context window, ChatGPT usually has no problem remembering what it's talking about. This isn't GPT 2, which wandered all over the place incoherently. ChatGPT is capable of writing a 1-page short story that is consistent throughout, and that has a clear structure and progression.
e.g. ask it for instructions on how to commit a crime, it will tell you its ethical guidelines prohibit it from answering. Ask it for more detail on the contents of those ethical guidelines, it will tell you that as a large language model, it is incapable of possessing any such thing as ethical guidelines. Point out it just contradicted itself, it is likely to respond that as a large language model it is incapable of contradicting itself, or else give some non-sensical explanation as to why the two statements aren’t contradictory
Ask it if it agrees with what you just said, it might reply that yes it does. Ask it again, it may instead give you a lengthy lecture on how as a large language model it isn’t capable of agreeing with anything.
I suspect it just wasn’t trained enough on the importance of self-consistency; nor on the principle that it is better to honestly admit to contradicting yourself than to give a non-sensical defence of your own contradictions
I even wonder if some of its training may have actually been contradictory - if trainers individually reward it both for saying “my ethical guidelines don’t permit me to answer that”, but also “I’m not capable of having ethical views” when pressed for judgements on ethical matters.