Technically this is possible with these same techniques if you just initialize the image with the prior one, though I am sure that does not work that well.
Really you need image+text->image instead of just text->image generation. Some examples of relevant papers: "Conditioned and composed image retrieval combining and partially fine-tuning CLIP-based features", "IMAGE GENERATION WITH MULTI-MODAL PRIORS USING DENOISING DIFFUSION PROBABILISTIC MODELS". There was a more recent one I saw on Twitter I don't recall the name of. I wouldn't be surprised if these kinds of things work well by a year from now.