ControlNet Speed Performance Collection

[lllyasviel]

Because SDXL is a relatively large model, we need to make sure that the current codes are correctly using your GPU.

We invite you to share some screenshots like this from your webui here:

The “time taken” will show how much time you spend on generating an image. The “sys” will show the VRAM of your GPU.

We highly appreciate your help if you can share a screenshot in this format:

GPU (like RGX 4096, RTX 3080, ...)
-cmdflag (like --medvram-sdxl, --lowvram, ...)
Then put an image

(To avoid the influence of downloading preprocessors, we recommend to use canny for test. If possible, you may use 1024 resolution and sdxl models. Feel free to use any canny control model as you wish.)

I will start with my own test as an example. Looking forward to your test screenshots.

[lllyasviel]

RTX 3070ti laptop GPU
--lowvram --xformers

[lllyasviel]

RTX 3070ti laptop GPU
--medvram-sdxl --xformers

[lllyasviel]

RTX 3070ti laptop GPU
--medvram-sdxl --xformers

[lllyasviel]

RTX 3070ti laptop GPU
--medvram-sdxl --xformers

[lllyasviel]

RTX 4090
--xformers

[lllyasviel]

3060 laptop GPU
--lowvram --xformers

[youyegit]

RTX 3080 12G
--xformers --medvram-sdxl

[w-e-w]

RTX 3090 24G --xformer

[manticorp]

RTX 4090 24GB

--opt-sdp-no-mem-attention --autolaunch --gradio-img2img-tool color-sketch

[manticorp]

RTX 4090 24GB

--opt-sdp-no-mem-attention --autolaunch --gradio-img2img-tool color-sketch

[manticorp]

RTX 4090 24GB

--opt-sdp-no-mem-attention --autolaunch --gradio-img2img-tool color-sketch

[HMRMike]

RTX 3090 24GB
--xformers

[lllyasviel]

Sep 6:
I looked at all above. All cases are within expected speed.
Please share more screenshots to help us investigating these cases

[ClashSAN]

RTX 3050 laptop GPU
--opt-sdp-no-mem-attention --lowvram --theme dark

[JetsprintRacer]

RTX 3080 Ti
For some reason this thing is slow as snail when I use SDXL Reference, and "OutOfMemoryError: CUDA out of memory" when I try to use Sargezt XL Softedge. Estimated time is 30-60 minutes sometimes. How in the world you managed to render pictures in 36 seconds without any problems while using a Reference model - that's quite a mistery for me. Also, a lot of "Out of memory errors", when I'm interrupting generations too often - probably, allocated memory unload problem. Revision, IP-Adapter and Sai Canny seem to work fine. I guess... Also, first time generation with new preprocessor image in SDXL is quite long.

[lllyasviel]

-cmdflag is missing. This problem should be solved by adding --medvram-sdxl

[gumpngreen]

What's the difference between --medvram and --medvram-sdxl?

[JetsprintRacer]

First one affects all checkpoints, second one affects only SDXL checkpoints.

[boranali]

rtx 1050 4 GB laptop :D

cinematic still photography of a perfect modern sports car. best quality,high resolution,raw photo,designed by a master,flawless. emotional,harmonious,vignette,highly detailed,high budget,bokeh,cinemascope,moody,epic,gorgeous,film grain,grainy,

Negative prompt: drawing,painting,illustration. worst quality,low quality,low resolution. imperfect,bad design. anime,cartoon,graphic,rendered,text,crayon,graphite,abstract,glitch,deformed,mutated,ugly,disfigured,
Steps: 40, Sampler: DPM++ 2M SDE, CFG scale: 7, Seed: 3836892232, Size: 1024x1024, Model hash: 0f1b80cfe8, Model: dreamshaperXL10_alpha2Xl10, RNG: CPU, SGM noise multiplier: True, Version: v1.6.0

Time taken: 9 min. 11.9 sec.

A: 3.41 GB, R: 7.14 GB, Sys: 4.0/4 GB (100.0%)

[MoonRide303]

RTX 4080 (desktop), base + refiner, double Control-LoRA (rank 128, canny + depth).

COMMANDLINE_ARGS:

--xformers --medvram --no-half-vae

Prompt:

cinematic still perfect supercar . emotional, harmonious, vignette, highly detailed, high budget, bokeh, cinemascope, moody, epic, gorgeous, film grain, grainy
Negative prompt: anime, cartoon, graphic, text, painting, crayon, graphite, abstract, glitch, deformed, mutated, ugly, disfigured
Steps: 30, Sampler: DPM++ 2M SDE Karras, CFG scale: 7, Seed: 0, Size: 1248x832, Model hash: e6bb9ea85b, Model: sd_xl_base_1.0_0.9vae, Clip skip: 2, RNG: CPU, ControlNet 0: "Module: canny, Model: control-lora-canny-rank128 [c910cde9], Weight: 0.5, Resize Mode: Resize and Fill, Low Vram: False, Processor Res: 1024, Threshold A: 100, Threshold B: 200, Guidance Start: 0, Guidance End: 0.3, Pixel Perfect: False, Control Mode: Balanced", ControlNet 1: "Module: none, Model: control-lora-depth-rank128 [df51c1c8], Weight: 0.5, Resize Mode: Resize and Fill, Low Vram: False, Guidance Start: 0, Guidance End: 0.3, Pixel Perfect: False, Control Mode: Balanced", SGM noise multiplier: True, Refiner: sd_xl_refiner_1.0_0.9vae [8d0ce6c016], Refiner switch at: 0.67, Version: v1.6.0

Results, 1st run:

Time taken: 36.8 sec.

A: 6.92 GB, R: 12.52 GB, Sys: 15.0/15.9922 GB (93.8%)

Results, 2nd run:

Time taken: 27.7 sec.

A: 6.95 GB, R: 12.55 GB, Sys: 15.1/15.9922 GB (94.1%)

Input (same to both CNs):

Output:

[kasimgu2000]

Not sure whats going on here....any ideas?

RTX 3080 Ti 12GB

--medvram-sdxl --xformers --no-half --autolaunch

[VeraLapsa]

I'm on a 3070-8GB and if I run the latest nvidia drivers (I think 536.99) my generation time skyrockets because of the memory management it does (offloads to system memory). I don't know if it'll help but downgrading to the 531.61 drivers fixed long generation times for me.

[MoonRide303]

536.67 work fine for me.

[kasimgu2000]

no luck with changing drivers....might try a complete re-install of everything...

[MoonRide303]

@kasimgu2000 You can also try with different COMMANDLINE_ARGS in webui-user.bat, these work reasonably for me: set COMMANDLINE_ARGS=--xformers --medvram --no-half-vae.

[kasimgu2000]

MoonRide303 - Yes! This particular combination of parameters seem to have worked! Thank you =)
Now for the next challenge: trying to get the Lineart and Reference Controlnets actually do something...