[diffusion] Enable Cache‑DiT config for diffusers backend#16662
Merged
mickqian merged 7 commits intosgl-project:mainfrom Jan 22, 2026
Merged
[diffusion] Enable Cache‑DiT config for diffusers backend#16662mickqian merged 7 commits intosgl-project:mainfrom
mickqian merged 7 commits intosgl-project:mainfrom
Conversation
github-actions
Bot
added
documentation
Contributor
Summary of ChangesHello @qimcis, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces support for Cache-DiT acceleration within SGLang's Diffusers backend. It allows users to leverage Cache-DiT for optimizing any Diffusers pipeline by providing a configuration file. The changes involve adding a new command-line argument to specify the Cache-DiT configuration, integrating the Cache-DiT enablement logic directly into the Diffusers pipeline loading process, and updating relevant documentation to guide users on this new feature. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces support for Cache-DiT acceleration within the diffusers backend. It adds a new CLI argument --cache-dit-config to specify a Cache-DiT configuration file, updates the ServerArgs dataclass to include this configuration, and integrates the Cache-DiT enabling logic into the DiffusersPipeline.
The changes also include minor documentation updates to reflect the new CLI argument and clarify limitations regarding distributed support for Cache-DiT.
Comment on lines
+556
to
+558
| except Exception: | ||
| logger.exception("Failed to enable cache-dit for diffusers pipeline") | ||
| raise |
Contributor
There was a problem hiding this comment.
The try...except Exception block is very broad. While logger.exception is helpful for debugging, catching a generic Exception can mask specific issues that might arise from cache_dit.enable_cache. Consider catching more specific exceptions if known, or at least adding a comment explaining why a broad exception is necessary here (e.g., due to the external nature of the cache_dit library and its potential to raise various exceptions).
Comment on lines
+643
to
+644
| except Exception: | ||
| pass |
Contributor
There was a problem hiding this comment.
Catching a broad Exception and then passing can hide critical errors, making debugging difficult. If get_text_encoder_from_pipe can fail in expected ways, consider catching those specific exceptions. Otherwise, it's better to log the exception at a debug level or re-raise it if it indicates a serious problem, rather than silently ignoring it.
Collaborator
|
@DefTruth could you take a look when you're available? |
Contributor
cool~ I'll review it this afternoon |
Contributor
|
@qimcis @mickqian Thanks for this great work! I have some suggestions as follows:
# Quantization will be treated as a third-party tool for cache-dit, so we do not plan to
# support quant_config for this backend. For quantization-related use cases, we
# recommend directly using sglang (instead of the cache-dit backend) as it delivers
# better performance.
cache_dit.enable_cache(pipe, **cache_dit.load_configs("cache_dit_config.yml"))
|
Contributor
|
@qimcis The docs for |
mickqian
reviewed
Jan 9, 2026
| help="Attention backend for diffusers pipelines (e.g., flash, _flash_3_hub, sage, xformers). " | ||
| "See: https://huggingface.co/docs/diffusers/main/en/optimization/attention_backends", | ||
| ) | ||
| parser.add_argument( |
Collaborator
There was a problem hiding this comment.
this might function well. but we use env vars for non-diffusers-backend, introducing such a new arg could cause confusion
qimcis
requested review from
Fridge003,
ispobock and
merrymercy
as code owners
github-actions
Bot
added
the
dependencies
Contributor
|
🤗Cache-DiT v1.2.0 Major Release is ready! I recommend that we use this version. |
Contributor
Author
|
Apologies for the delay, have had a busy week I think I've implemented all the requested changes, lmk if I missed anything After rerunning the tests on 1.2.0 I'm seeing good improvements! Let me know if these are approximately expected numbers @DefTruth
|
Contributor
Hi~ thank you for providing the test data. The overall result is LGTM. Could you please also provide the complete test log at the same time? I can help analyze what causes the difference in the acceleration ratio. |
Contributor
Author
Yup! Attached below: Test 1 BaselineTest 1 Cache-DiTTest 2 BaselineTest 2 Cache-DiT |
Contributor
|
can you also share the test logs based on cache-dit v1.1.8 ? |
Contributor
Author
I don't have the logs for when I originally ran the tests, I just reran them on the previous commit, logs below: Test 1 BaselineTest 1 Cache-DiTTest 2 BaselineTest 2 Cache-DiT |
Contributor
|
@qimcis @mickqian From the comparison of the time consumption in Transformers, I believe the results are as expected because Cache-DiT only affects the Transformer modules. What do you think? cache-dit v1.1.8 # Test 1 Cache-DiT, Transformer ~ 17s
100%|█████████████████████████████████████████████████████████| 50/50 [00:17<00:00, 2.93it/s]
100%|█████████████████████████████████████████████████████████| 50/50 [00:17<00:00, 2.93it/s]
# Test 2 Cache-DiT, Transformer ~ 16s
100%|█████████████████████████████████████████████████████████| 50/50 [00:16<00:00, 2.99it/s]
100%|█████████████████████████████████████████████████████████| 50/50 [00:16<00:00, 2.99it/s]cache-dit v1.2.0 # Test 1 Cache-DiT, Transformer ~ 17s
100%|█████████████████████████████████████████████████████████|50/50 [00:17<00:00, 2.94it/s]
100%|█████████████████████████████████████████████████████████|50/50 [00:17<00:00, 2.94it/s]
# Test 2 Cache-DiT, Transformer ~ 16s
100%|█████████████████████████████████████████████████████████| 50/50 [00:16<00:00, 2.99it/s]
100%|█████████████████████████████████████████████████████████| 50/50 [00:16<00:00, 2.99it/s] |
Contributor
Author
this looks like it could be the case! |
Contributor
|
@mickqian do you have time to take a look? thanks~ |
Contributor
|
@qimcis hi, can you also test the configs w/o parallel text encoder and parallel vae for this latest commit? Just remove |
mickqian
approved these changes
Jan 21, 2026
Collaborator
|
/tag-and-rerun-ci |
Contributor
Author
tested most recent commit on cache dit 1.2.0 w/o parallel text encoder and parallel vae, with added --warmup True flag: |
mickqian
force-pushed
the
cache-dit-diffusers
branch
from
e2bec4d to
073f328
Compare
Contributor
Author
should i add tests in this pr? or make a separate pr for these |
Collaborator
|
@qimcis you could do it later |
Collaborator
|
diffusion tests and lint all passed, the failed srt tests are irrelevant |
mickqian
approved these changes
Jan 22, 2026
Johnsonms
pushed a commit
to Johnsonms/sglang
that referenced
this pull request
Feb 14, 2026
…nd (sgl-project#16662) Signed-off-by: Chi <chixie.mcisaac@gmail.com> Signed-off-by: qimcis <chixie.mcisaac@gmail.com> Co-authored-by: Mick <mickjagger19@icloud.com>
5 tasks
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Addressing #16642, enabling Cache‑DiT acceleration for any diffusers pipeline in SGLang by allowing a cache‑dit config file to be passed through the diffusers backend.
Modifications
--cache-dit-configserver arg inpython/sglang/multimodal_gen/runtime/server_args.pypython/sglang/multimodal_gen/runtime/pipelines/diffusers_pipeline.py.--diffusers-kwargspath to avoidSamplingParamsinit errors:python/sglang/multimodal_gen/configs/sample/sampling_params.py.Accuracy Tests
Benchmarking and Profiling
2x RTX Pro 6000 WS
Test 1
Test 2
Checklist