Skip to content

[Diffusion][Cosmos3] Enable breakable CUDA graph - #34618

Closed
BBuf wants to merge 1 commit into
sgl-project:mainfrom
BBuf:bbuf/b300-cosmos3-bcg
Closed

BBuf wants to merge 1 commit into
sgl-project:mainfrom
BBuf:bbuf/b300-cosmos3-bcg

Conversation

@BBuf

@BBuf BBuf commented Aug 12, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Enable breakable CUDA graph for Cosmos3 Nano and preserve request warmup for request-only dimensions such as frame count.

The Cosmos-specific prompt padder keeps max_text_seq_len consistent with the padded text mask. Warmup captures the requested text buckets before serving and forwards the current request context through the graph runner.

B300 result

Cosmos3 Nano T2I, 1024px, 35 steps, native backend:

mode denoise e2e
eager + BCG 0.880527 s 0.923572 s
torch.compile 0.908647 s 0.952747 s

BCG is 3.09% faster in denoise and 3.06% faster end to end.

Validation

  • B300 Cosmos/BCG/server-args: 256 passed, 46 subtests passed
  • pre-commit: passed

Performance was measured on commit 9deb6952afa483e38f96385a375b96f463da5303; this PR was rebased and retested on the latest main.


CI States

Latest PR Test (Base): ✅ Run #31628635294
Latest PR Test (Extra): 🚫 Run #31630111122

@github-actions github-actions Bot added the diffusion SGLang Diffusion label Aug 12, 2026
@BBuf BBuf added run-ci CI: run the baseline test suite on this PR run-ci-extra CI: also run the extra suite (requires run-ci) labels Aug 12, 2026
@BBuf

BBuf commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator Author

Closing this candidate after the latest review. The Cosmos3 BCG path improved the measured B300 denoise latency, but the request-warmup lifecycle and model-specific state/test surface are not clean enough to justify the added maintenance risk. I will restore the B300 table to the retained main-branch result and mark this PR as closed.

@BBuf BBuf closed this Aug 13, 2026
@BBuf
BBuf deleted the bbuf/b300-cosmos3-bcg branch August 13, 2026 08:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

diffusion SGLang Diffusion run-ci CI: run the baseline test suite on this PR run-ci-extra CI: also run the extra suite (requires run-ci)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant