Hacker News Reader

I suspect it’s not that similar. The intuition behind LoRA is more true the higher the rank of the weights of the model. Even the smallest LLMs have considerably higher rank weights than Stable Diffusion. They are large, after all.

https://dreambooth.github.io/

The LoRA paper’s ‘problem statement’ makes a compelling case for practical benefits of the approach. Specifically, no added latency, no serial processing bottlenecks, shared baseline model, compact time/space requirements. How does dreambooth stack up in this regard?

In the image space, dreambooth full-model tunes can handle multiple concepts and tend to be easier to get hard/complex things like a person’s likeness correct. I’ve found that LoRA tunes struggle to be accepted by people as producing their own face compared to full dreambooth models tuned on the same inputs, most likely because we are very sensitive to facial differences of faces we are very familiar with. I haven’t seen this effect for styles or other kinds of concepts, where people are a little less sensitive about the fidelity. LoRA is much easier to train, easier to compose, and can have the base model swapped out in many cases though so if it’s good enough for the concept you are trying to add to the model it’s often worth the subtle quality loss.

There are some WIP evolutions of SD Lora in the works, like locon and lycoris.

https://github.com/KohakuBlueleaf/LyCORIS