At least, we know it's capable of generating readable text, which SD sure isn't, and there are newer model papers out there like DiT that should beat latent diffusion on quality and speed.
At least, we know it's capable of generating readable text, which SD sure isn't, and there are newer model papers out there like DiT that should beat latent diffusion on quality and speed.
Also, I'd hold my horses with calling latent diffusion "out of date". The Imagen paper notes that the results greatly scaled with text-encoder size and the model DeepFloyd (it's a team) are working on uses T5-XXL (the unet is also a decent bit larger, but the Imagen paper claims that this should have a much smaller influence). Latent diffusion might also be able to work with text if you scale up the text encoder.
I'm very skeptical of Google's "trust me, I have the hottest model, it just goes to another school". Muse might have nice fidelity and be fast, but the example images have worse visual quality than Imagen. I definitely wouldn't be surprised if transformers took over, but I don't think it's a foregone conclusion.
Your advanced model becomes a toy, like creating a Twitter clone on your local machine.
You then have to recreate all these things like control net, because I know for sure that this fine tuned control is way more important than a jump in base model quality.
This is why open source kicks ass. The SD people don't have to be the best, not with a community floating them.