Skip to content

[diffusion] cli: support component attention backend overrides#24320

Merged
mickqian merged 7 commits into
sgl-project:mainfrom
mickqian:codex/component-attn-backend-20260504
May 5, 2026
Merged

[diffusion] cli: support component attention backend overrides#24320
mickqian merged 7 commits into
sgl-project:mainfrom
mickqian:codex/component-attn-backend-20260504

Conversation

@mickqian
Copy link
Copy Markdown
Collaborator

@mickqian mickqian commented May 3, 2026

Summary

  • Add --component-attention-backends for per-component native attention backend overrides.
  • Apply overrides while pipeline-defined modules/components are constructed, without hardcoding model-specific components.
  • Carry component name through attention backend selection logs.
  • Document the new CLI/API usage in both docs trees.

Test

  • git diff --check
  • Local unittest collection is blocked on this Mac by ImportError: cannot import name '_cuda_beginAllocateCurrentThreadToPool' from 'torch.cuda.memory'.

@github-actions github-actions Bot added documentation Improvements or additions to documentation diffusion SGLang Diffusion labels May 3, 2026
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the --component-attention-backends CLI argument, allowing users to specify per-component attention backend overrides (e.g., text_encoder=torch_sdpa). The implementation includes updates to the attention selector to support ContextVar-based overrides, integration into component loaders and pipeline construction, and logic in ServerArgs to parse both comma-separated strings and dotted CLI arguments. Documentation and unit tests have been updated accordingly. I have no feedback to provide.

@mickqian mickqian changed the title Add component attention backend overrides [diffusion] feat: support component attention backend overrides May 3, 2026
@mickqian mickqian changed the title [diffusion] feat: support component attention backend overrides [diffusion] cli: support component attention backend overrides May 3, 2026
@mickqian mickqian marked this pull request as ready for review May 3, 2026 17:20
@mickqian
Copy link
Copy Markdown
Collaborator Author

mickqian commented May 3, 2026

/tag-and-rerun-ci

@github-actions github-actions Bot added the run-ci label May 3, 2026
@mickqian mickqian merged commit 2f7d99b into sgl-project:main May 5, 2026
73 of 79 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

diffusion SGLang Diffusion documentation Improvements or additions to documentation run-ci

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant