Skip to content

[Bugfix] [diffusion] Fix cache-dit with sp-degree only #19965

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
BBuf merged 10 commits into from
Mar 18, 2026
Merged

[Bugfix] [diffusion] Fix cache-dit with sp-degree only #19965

Changes from 2 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
6e9e773
Update cache_dit_integration.py
OrangeRedeng Mar 5, 2026
18da174
Fix lint issue
OrangeRedeng Mar 5, 2026
9c83b81
Merge branch 'main' into sequence_parallelism_with_cache_fix
OrangeRedeng Mar 8, 2026
2f85c55
Merge branch 'main' into sequence_parallelism_with_cache_fix
OrangeRedeng Mar 11, 2026
78f1940
Switch to built-in functions
OrangeRedeng Mar 11, 2026
6f0d25a
Update cache_dit_integration.py
OrangeRedeng Mar 11, 2026
fab79fd
Update cache_dit_integration.py
OrangeRedeng Mar 11, 2026
11f2904
Merge branch 'main' into sequence_parallelism_with_cache_fix
mickqian Mar 15, 2026
bc8049b
Merge branch 'main' into sequence_parallelism_with_cache_fix
OrangeRedeng Mar 16, 2026
0dc8ed3
Merge branch 'main' into sequence_parallelism_with_cache_fix
ping1jing2 Mar 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions python/sglang/multimodal_gen/runtime/cache/cache_dit_integration.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
import torch
import torch.distributed as dist

from sglang.multimodal_gen.runtime.distributed.parallel_state import get_sp_group
from sglang.multimodal_gen.runtime.utils.logging_utils import init_logger

logger = init_logger(__name__)
Expand Down Expand Up @@ -107,15 +108,15 @@ def _build_parallelism_config(
ulysses_size = None
ring_size = None
if sp_group is not None:
ulysses_size = getattr(sp_group, "ulysses_world_size", None)
ring_size = getattr(sp_group, "ring_world_size", None)
ulysses_size = get_sp_group().ulysses_world_size

@DefTruth DefTruth Mar 6, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just use sp_group? It's no longer None.

@OrangeRedeng OrangeRedeng Mar 6, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure? I test it now and it still doesn't work on the latest version of slang from the main

@OrangeRedeng OrangeRedeng Mar 6, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which PR fix this issue?

@OrangeRedeng OrangeRedeng Mar 6, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

@OrangeRedeng OrangeRedeng Mar 6, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my case (and #19955) the sp_group has no attributes "ulysses_world_size" and "ring_world_size"

@DefTruth DefTruth Mar 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my case (and #19955) the sp_group has no attributes "ulysses_world_size" and "ring_world_size"

Wired, @BBuf could you please also take a look? thanks~

@DefTruth DefTruth Mar 9, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my case (and #19955) the sp_group has no attributes "ulysses_world_size" and "ring_world_size"

@OrangeRedeng Hi~ I think you are right, the sp/tp_group inside _build_parallelism_config are assumed to be torch.distributed.ProcessGroup, but ProcessGroup has no attributes "ulysses_world_size" and "ring_world_size". The bug #19955 you has encountered maybe cause by errored sp/tp_group assigning logics at:

sglang/python/sglang/multimodal_gen/runtime/pipelines_core/stages/denoising.py

Lines 205 to 206 in 484f53c

sp_group = sp_group_candidate.device_group if has_sp else None
tp_group = tp_group_candidate.device_group if has_tp else None

Maybe we should change these code snippets to:

 sp_group = sp_group_candidate if has_sp else None 
 tp_group = tp_group_candidate if has_tp else None

By the way, we should also fix the signature of _build_parallelism_config to avoid misleading developers with the wrong usage. From the old one:

sglang/python/sglang/multimodal_gen/runtime/cache/cache_dit_integration.py

Lines 100 to 103 in 484f53c

def _build_parallelism_config(
sp_group: Optional[torch.distributed.ProcessGroup],
tp_group: Optional[torch.distributed.ProcessGroup],
):

to the correctly signature:

def _build_parallelism_config( 
     sp_group: Optional[GroupCoordinator], 
     tp_group: Optional[GroupCoordinator], 
 ):

@OrangeRedeng Could you please take a try?
Also cc @mickqian

ring_size = get_sp_group().ring_world_size

@gemini-code-assist gemini-code-assist Bot Mar 5, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To improve readability and avoid calling get_sp_group() twice, you can store its result in a local variable and reuse it.

Suggested change
ulysses_size = get_sp_group().ulysses_world_size
ring_size = get_sp_group().ring_world_size
sp_coord = get_sp_group()
ulysses_size = sp_coord.ulysses_world_size
ring_size = sp_coord.ring_world_size


tp_size = None
if tp_group is not None:
tp_size = dist.get_world_size(tp_group)

return ParallelismConfig(
backend=ParallelismBackend.NATIVE_PYTORCH,
backend=ParallelismBackend.AUTO,
ulysses_size=ulysses_size,
ring_size=ring_size,
tp_size=tp_size,
Expand Down
Loading