Enable Convolution AutoTuning

[contentis]

This will test the Top10 algorithms returned by the cuDNN heuristic and select the fastest. On a 5090 I'm seeing

• 1.41x on SDXL
• 1.32x on SD15
• 1.21x on VAE Decoder (SDXL)

This does add a small overhead during the first inference - in my case this was ~200ms.

[comfyanonymous]

The overhead in my case during first inference on blackwell 6000 is ~15 seconds for an 1280x1680 SDXL workflow. Will have to decide if this should be enabled by default or behind --fast

[contentis]

Was this on Windows or Linux? I retested on an RTX6000, SDLX @ 1280x1680, Ubuntu 24.04 and saw the following:

SDXL

• With AutoTuning
  • First Run: 6690.49ms
  • Second Run 3168.08ms
• Without
  • First Run: 5524.88ms
  • Second Run: 4161.28ms

One thing I noticed is that the alloc backend has a large influence on this - I was using native for all my testing. When using cuda-malloc the first run takes ~12006.10ms

[contentis]

I added an option to the --fast autotune argument to make this feature opt-in for now.

[comfyanonymous]

can you rebase? there's something weird with your branch.

[Arcitec]

Sounds like a very good improvement. If it adds up to ~5 - 10 seconds on the first inference, that's not worth worrying about and should be on by default, since it saves so much time in general. People will make up that time after 2 images.

But it seems like it adds less than a second to the first run if a proper model loading backend is used. I guess the reason for longer delays is that cuda-malloc has some overhead that really adds up with repeated tests?

PS: The pull request shows a ton of changes. Needs a rebase. :')

[contentis]

It was already re-based, github had issues updating the PR - I reset the target branch, it should now show up correctly.

[Arcitec]

Thank you, I can finally see the diff now. :) Oh, so cuDNN benchmarking is a built-in PyTorch feature. That's awesome. I'll try to make time to test this on my 3090 on Linux (the 5090 arrives this week so I can try both). I'm working on a lot of projects and my Comfy is currently outdated. I'll try to make time for it!

Edit: I didn't have time to test it before merge, oops. I was super busy. :')

[FeepingCreature]

The overhead is a lot higher on ROCm, at least my 7900 XTX, taking over a minute to tune a basic SDXL kernel. So this should probably be turned off on AMD unless explicitly requested.

[Windecay]

if use autotune...task start very slow. On rtx 5090 prepare 10s sampling 3s. aha.