Stable Diffusion Gets a Major Boost with RTX Acceleration | Hacker News Reader

Stable Diffusion Gets a Major Boost with RTX Acceleration

nvidia.com

101 pointsortusdux26 comments

Open articleView on HN

Before you jump into installing the extension for your A1111, keep in mind the limitations:

- You have to "pre-bake" each model (create optimized engines) for only a pre-defined set of resolutions

- ~~No LoRA support~~ I guess there is a LoRA support, but you also will need to convert them to TRT

It's been like this for quite a few months now, not sure what's new this time.

Why is LoRA not supported? Isn't it just an additional two matrix multiplications per layer?

You have to convert them and you need to adept the model loader in your tooling.

LoRA are explicitly supported, though you have to convert them, as well as the base model.

This seems likely to be more useful for people with a "production" workflow with particular checkpoint, LoRA, & resolution combo than what I get the impression is the more common hobbyist situation of having a decently diverse collection of checkpoints and LoRA and fairly freely mixing and matching.

Played around with it, only a few dozen images so far, and the difference is pretty noticeable.

On a 2060S, no launch arguments, going from ~4 it/s to ~7.5 it/s.

Link to the support page: https://nvidia.custhelp.com/app/answers/detail/a_id/5487

Thanks! Can't wait to get a minute to check this out. Promising 2x performance over pytorch+xformers sounds too good to be true for the same card.

As someone with a lowly 10gb card sdxl is beyond my reach with a1111 it seems. It functions well enough in comfyui but I can't make anything but garbage with it in automatic. So I'm happy to see 1.5 gets a big boost, I know there's a million of us out there who can't quite squeeze SDXL out so the maturing of the "legacy" versions is a positive note to see.

I haven't been paying attention for a few months, is there a lot of development happening around SDXL or are people still mostly focused on 1.5? My expectation was that SDXL would probably not see wide adoption for a while because it's hard to make up for the worse training data, is that bearing out?

>My expectation was that SDXL would probably not see wide adoption for a while because it's hard to make up for the worse training data

Where do you read that SDXL has worse training data? They didn't disclose what the model was trained on like the previous models.

Most people (and models) still use 1.5 because of hardware and performance issues with SDXL.

> I haven't been paying attention for a few months, is there a lot of development happening around SDXL or are people still mostly focused on 1.5?

There's been a lot happening on SDXL, yes.

> My expectation was that SDXL would probably not see wide adoption for a while because it's hard to make up for the worse training data, is that bearing out?

What worse training data?

> There's been a lot happening on SDXL, yes.

Nice! Care to share any projects/advancements/gens you've found particularly compelling if you have anything handy?

> What worse training data?

I don't know this, but I think it's a pretty safe assumption given the state of 2.0/the political climate around this stuff, along with the comparative lower diversity of gens I've seen come out of SDXL. I haven't seen anything to the contrary but I'd love to be wrong.

I've used SDXL with a 3080 (10GB) mostly without issues. You have to give some command line parameters when launching it to reduce memory use.

> --medvram-sdxl --no-half-vae

"--no-half-vae" doesn't reduce memory usage, it forces A1111 to use 32-bit floats for the VAE decoding.

--no-half-vae, IIRC, is required for the stock sdxl vae model because otherwise it frequently produces errors. I believe there’s a half-precision fixed VAE floating around and baked into many SDXL-derived checkpoints that avoids this problem.

Runs just fine on my 8GB card. You probably did not swap the VAE. Without that it will only produce junk.

> As someone with a lowly 10gb card sdxl is beyond my reach with a1111 it seems

SDXL reportedly works at below 8GiB in A1111 with --low-vram, and at 8GiB to anything short of 16GiB with --med-vram-sdxl, and 16GiB and up with no options.

> but I can't make anything but garbage with it in automatic.

If you aren't getting out of memory errors with SDXL but the output sucks, its probably not a VRAM problem. The most likely thing, IME, is using the SDXL base model or another checkpoint without a baked VAE but still having A1111 configured to use an SD1.x VAE (or using one with a baked VAE but having A1111 configured to override it.) SDXL needs an SDXL-specific VAE, but A1111 doesn't bundle one, IIRC, you need to download it separately.

Its cool that this is starting to approach real time video territory (30 images per second, this claims close to 1 image/sec).

Once you reach a few fps, you can use other techniques to interpolate frames.

SD is trained on static images and don't have a clue about motion. Interpolation won't help with that. See deforum videos. They don't make any sense. Fast generation will help iterate faster though.

Here's a direct link to the extension that's referenced in the page. https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT

For developers, TensorRT support also exists for the diffusers library via community pipelines. [1] It's limited, but if you're only supporting a subset of features, it can help.

In general, these insane speed boosts come at the cost of bleeding edge features.

[1] https://github.com/huggingface/diffusers/blob/28e8d1f6ec82a6...

Useless. I don't see instructions neither for Linux nor for Mac. + there are no drivers for Linux or Mac.

I want to make an RTX on joke but my card isn't supported...

Mr. Miyagi: RTX on, RTX off...

Legit can't wait to generate images even faster.

Reply on news.ycombinator.com