[diffusion] Add cache-dit CI tests by qimcis · Pull Request #19213 · sgl-project/sglang

qimcis · 2026-02-24T04:02:43Z

Motivation

Add CI tests for changes in #16662 , and a CI test for cache-dit on native diffusion engine as well

Modifications

Add 1 gpu test, 2 gpu test and baselines

Checklist

Format your code according to the Format code with pre-commit.
Add unit tests according to the Run and add unit tests.
Update documentation according to Write documentations.
Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
Follow the SGLang code style guidance.

gemini-code-assist · 2026-02-24T04:03:02Z

Summary of Changes

Hello @qimcis, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request integrates new Continuous Integration (CI) tests for the cache-dit diffusers feature, covering both single and dual GPU setups. The primary goal is to ensure the stability and performance of this caching mechanism within the diffusers backend by establishing dedicated test cases and updating performance baselines. This enhancement improves the robustness of the multimodal generation server by validating critical configurations.

Highlights

New CI Tests: New CI tests for cache-dit diffusers functionality were introduced, covering both 1-GPU and 2-GPU configurations to ensure stability and performance.
Configuration Files Added: Dedicated YAML configuration files (cache_dit_config_1gpu.yaml, cache_dit_config_2gpu.yaml) were added to define specific cache-dit settings for the new tests.
Performance Baselines Updated: Performance baselines were updated to include expected metrics for the newly added cache-dit diffusers test cases.
Test Framework Extension: The test framework was extended to support passing diffusers_kwargs to the image generation client, allowing for more flexible testing of diffusers-specific parameters.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

python/sglang/multimodal_gen/test/server/cache_dit_config_1gpu.yaml
- Added a new YAML configuration file for 1-GPU cache-dit settings.
python/sglang/multimodal_gen/test/server/cache_dit_config_2gpu.yaml
- Added a new YAML configuration file for 2-GPU cache-dit settings, including parallelism parameters.
python/sglang/multimodal_gen/test/server/perf_baselines.json
- Updated performance baselines by adding entries for new qwen_image_t2i_cache_dit_config_diffusers_1gpu and qwen_image_t2i_cache_dit_config_diffusers_2gpu test cases.
python/sglang/multimodal_gen/test/server/test_server_utils.py
- Modified the generate_image function to pass diffusers_kwargs from sampling parameters to the image generation client.
python/sglang/multimodal_gen/test/server/testcase_configs.py
- Added a diffusers_kwargs field to the DiffusionSamplingParams dataclass.
- Introduced new DiffusionTestCase entries for qwen_image_t2i_cache_dit_config_diffusers_1gpu and qwen_image_t2i_cache_dit_config_diffusers_2gpu, configuring them with specific cache-dit and diffusers_kwargs.

Activity

The author initiated this pull request to address the need for CI tests related to cache-dit diffusers functionality, as motivated by changes in [diffusion] Enable Cache‑DiT config for diffusers backend #16662.
The author has added new 1-GPU and 2-GPU test configurations along with their corresponding performance baselines.
The provided checklist indicates that code formatting, unit tests, and documentation updates have been completed.
The task to follow SGLang code style guidance is still pending.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request adds CI tests for cache-dit with the diffusers backend, covering both 1-GPU and 2-GPU scenarios. The changes are well-structured, including new YAML configurations, performance baselines, and test cases. The modification to pass diffusers_kwargs is appropriate. I have a couple of suggestions to improve code clarity and reduce duplication in the test case configuration.

gemini-code-assist · 2026-02-24T04:04:46Z

+        DiffusionSamplingParams(
+            prompt="Doraemon is eating dorayaki.",
+            output_size="1024x1024",
+            diffusers_kwargs={"max_sequence_length": 512},
+        ),


To avoid duplicating the prompt and output_size from T2I_sampling_params, you can use dataclasses.replace. This makes the code cleaner and ensures consistency with other test cases. It also makes it clear that this configuration is a variation of T2I_sampling_params.

replace(T2I_sampling_params, diffusers_kwargs={"max_sequence_length": 512}),

mickqian · 2026-02-25T01:40:53Z

@DefTruth is this PR covering sufficient cases? could you provide some suggestions? cheers

DefTruth · 2026-02-25T02:14:09Z

@DefTruth is this PR covering sufficient cases? could you provide some suggestions? cheers

@mickqian It would be better if some SCM-related tests could be added, e.g, cache_dit_scm_config.yaml.

mickqian · 2026-02-25T05:43:03Z

could you attach the output of this case?

qimcis · 2026-02-25T06:15:47Z

output:

DefTruth · 2026-03-10T03:55:46Z

@qimcis @mickqian Any updates? I think it would be better if we could add some tests for the cache to help SGLD avoid errors related to cache (e.g., #19955, #19965) introduced by new commits.

qimcis · 2026-03-10T03:58:53Z

@qimcis @mickqian Any updates? I think it would be better if we could add some tests for the cache to help SLGD avoid errors related to cache (e.g., #19955, #19965) introduced by new commits.

I can also help add more tests! Lmk if there are specific areas of coverage you think I should hit (besides the PR you mentioned already). Anything remaining I should address on this PR? @mickqian

mickqian · 2026-03-10T04:02:24Z

@@ -0,0 +1,10 @@
+cache_config:


gather to a configs folder?

mickqian · 2026-03-12T03:20:15Z

/tag-and-rerun-ci

yhyang201 · 2026-03-12T06:23:28Z

/rerun-failed-ci

yhyang201 · 2026-03-12T09:24:06Z

/rerun-failed-ci

yhyang201 · 2026-03-12T11:57:38Z

/rerun-failed-ci

yhyang201 · 2026-03-12T15:42:46Z

/rerun-failed-ci

yhyang201 · 2026-03-12T17:34:44Z

/rerun-failed-ci

qimcis · 2026-05-09T06:06:02Z

I updated this PR, instead of focusing only on cache-dit diffusers and the configs, I've also added a native SGLD 2-gpu sp-only cache-dit regression case, which uses the native SGLD cache-dit path with SP enabled and TP disabled, covering the class of issue from #19955 / #19965 around cache-dit parallelism config construction

In addition to this, i kept the 1-GPU diffusers cache-dit config coverage that i added previously (including one scm case)

Perf baselines have been regenerated onto current main too (2x h100 SXM)

Outputs for the 1gpu tests look like:

qwen_image_t2i_cache_dit_config_diffusers_1gpu

For the 2gpu test with wan, the output is not as important, as it's meant to measure simply that the execution path does not crash and produces a valid generation

Can you take a look to make sure things look ok before I get mick to review again? @DefTruth Thank you!

mickqian · 2026-05-09T08:50:17Z

        ),
        T2V_sampling_params,
    ),
+    DiffusionTestCase(


these tests are heavy, could we try to make it lightweight

i removed a redundant test, and made some changes, is this ok? passes locally for me @mickqian

mickqian · 2026-05-09T08:50:34Z

CI is failing

qimcis · 2026-05-09T17:06:02Z

/rerun-failed-ci

qimcis · 2026-05-09T23:33:36Z

/rerun-failed-ci

qimcis · 2026-05-10T01:34:07Z

/rerun-failed-ci

qimcis · 2026-05-10T03:12:36Z

/rerun-failed-ci

* main: (87 commits) [Fix] Disable FlashInfer allreduce fusion under deterministic inference (sgl-project#24629) fix: STANDALONE spec-decode hidden-size mismatch crash (sgl-project#24217) Followup fix for Custom AR V2 in non NVL scenarios (sgl-project#24742) Fix reduce_scatterv producer contract for SUM_LEN (sgl-project#24785) [NPU]Documentation update for communications quantization feature (sgl-project#24668) [Session R3] Add routed_experts_start_len for absolute routing slice control (sgl-project#24851) [Model] Add MiniCPM-V 4.6 support (sgl-project#24855) Support Intern-S2-Preview (sgl-project#24875) [PD] Unify dsv4 dispatch with swa (sgl-project#24888) Optimize MHC pipeline: DeepGemm, fused norm, fused hc_head (sgl-project#24775) Fix PD bootstrap failure handling (sgl-project#24772) [Spec] Cleanup idle stub and shape-check patterns (sgl-project#24881) [Bug] Add dsv4 state_type branch to mooncake disaggregation (sgl-project#24878) [Spec V1] Split draft-extend phase from `EagleDraftInput` into new `EagleDraftExtendInput` (sgl-project#24859) [Gemma4] Optimize Gemm4 with fused Q/K/V RMSNorm + per-expert FP8 ckpt loader (sgl-project#24696) [spec decoding] support kimi-k2.5-eagle3-mla (sgl-project#24826) [SPEC V2] fix: skip stale state updates in spec-v2 overlap (sgl-project#23456) [RL] Call torch.cuda.empty_cache() for `in-place` pause mode to avoid OOM (sgl-project#24854) [diffusion] CI: add cache-dit CI tests (sgl-project#19213) [Utils] Make request dump robust to unpicklable server_args and large meta_info (sgl-project#24767) ... # Conflicts: # python/sglang/srt/utils/common.py

qimcis requested review from mickqian, ping1jing2 and yhyang201 as code owners February 24, 2026 04:02

github-actions Bot added the diffusion SGLang Diffusion label Feb 24, 2026

gemini-code-assist Bot reviewed Feb 24, 2026

View reviewed changes

mickqian reviewed Mar 10, 2026

View reviewed changes

qimcis force-pushed the cachedit-test branch from b65c661 to 2559d8f Compare March 10, 2026 04:56

qimcis requested a review from mickqian March 10, 2026 05:09

github-actions Bot added the run-ci label Mar 12, 2026

qimcis mentioned this pull request May 7, 2026

[diffusion]: Fix diffusers executor crash when component residency manager is absent #24573

Merged

qimcis added 7 commits May 9, 2026 05:22

add

0593ced

fix

5c45e52

add

10ffede

fix

2ff693b

scm

018ba0b

fix baseline

57bbc8b

configs

6bad7e2

add

0c272f8

qimcis force-pushed the cachedit-test branch from 813c67b to 0c272f8 Compare May 9, 2026 05:55

qimcis changed the title ~~[diffusion] Add cache-dit diffusers CI tests~~ [diffusion] Add cache-dit CI tests May 9, 2026

mickqian approved these changes May 9, 2026

View reviewed changes

qimcis force-pushed the cachedit-test branch 2 times, most recently from 47dc984 to 6a5242c Compare May 9, 2026 16:33

fix

1819297

qimcis force-pushed the cachedit-test branch from 6a5242c to 1819297 Compare May 9, 2026 16:34

fix

17b1be2

Merge branch 'main' into cachedit-test

c4cbdae

mickqian merged commit 44efc23 into sgl-project:main May 10, 2026
248 of 283 checks passed

cursor Bot mentioned this pull request May 11, 2026

[AMD] Pin cache-dit==1.3.0 in rocm.Dockerfile + AMD CI install script #24924

Draft

5 tasks

Conversation

qimcis commented Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Modifications

Checklist

Uh oh!

gemini-code-assist Bot commented Feb 24, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

mickqian commented Feb 25, 2026

Uh oh!

DefTruth commented Feb 25, 2026

Uh oh!

mickqian commented Feb 25, 2026

Uh oh!

qimcis commented Feb 25, 2026

Uh oh!

DefTruth commented Mar 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

qimcis commented Mar 10, 2026

Uh oh!

mickqian Mar 10, 2026

Choose a reason for hiding this comment

Uh oh!

mickqian commented Mar 12, 2026

Uh oh!

yhyang201 commented Mar 12, 2026

Uh oh!

yhyang201 commented Mar 12, 2026

Uh oh!

yhyang201 commented Mar 12, 2026

Uh oh!

yhyang201 commented Mar 12, 2026

Uh oh!

yhyang201 commented Mar 12, 2026

Uh oh!

qimcis commented May 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mickqian May 9, 2026

Choose a reason for hiding this comment

Uh oh!

qimcis May 9, 2026

Choose a reason for hiding this comment

Uh oh!

mickqian commented May 9, 2026

Uh oh!

qimcis commented May 9, 2026

Uh oh!

qimcis commented May 9, 2026

Uh oh!

qimcis commented May 10, 2026

Uh oh!

qimcis commented May 10, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

qimcis commented Feb 24, 2026 •

edited

Loading

DefTruth commented Mar 10, 2026 •

edited

Loading

qimcis commented May 9, 2026 •

edited

Loading