Skip to content

[sglang-miles] Sync #40648: drop the DP-attention NCCL graph registration trigger - #41972

Merged
guapisolo merged 2 commits into
sglang-milesfrom
jiajun/sglang-miles-nccl-pausable-only
Oct 1, 2026
Merged

guapisolo merged 2 commits into
sglang-milesfrom
jiajun/sglang-miles-nccl-pausable-only

Conversation

@guapisolo

@guapisolo guapisolo commented Oct 1, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Stop disabling NCCL graph registration for DP-attention engines whose graph pool is not pausable.

Symptom & Reproduction

  • Symptom: every sglang-miles engine with --enable-dp-attention, --dp-size > 1 and decode CUDA graphs defaults to NCCL_GRAPH_REGISTER=0 even without --enable-memory-saver, so its graph-captured collectives lose NCCL zero-copy registration.
  • Reproduction: on sglang-miles 6e25d1bd57, test_without_all_to_all_env_is_untouched resolves a DP-attention config (tp_size=2, dp_size=2, default decode graphs) and expects NCCL_GRAPH_REGISTER to stay unset; it only holds because the test stubs _dp_attention_replays_decode_graphs to False. Without the stub, handle_data_parallelism sets it to 0.

Root Cause

  1. #40977 picked #40648 at 0854ad8c52, before review removed its DP-attention trigger.
  2. _dp_attention_replays_decode_graphs fires for every DP-attention engine that replays decode graphs.
  3. #40648 review found no hang from registration on a non-pausable pool, so upstream dropped it.

Fix

Two commits, so the swap is reviewable:

Colocated miles engines (offload rollout plus miles' default SGLANG_MEMORY_SAVER_CUDA_GRAPH=true) still get NCCL_GRAPH_REGISTER=0 with no miles change; DP-attention engines without a pausable pool return to NCCL's default of 1. An explicit NCCL_GRAPH_REGISTER still wins, with a warning.

Verification

Review Focus

  • _graph_pool_is_pausable / parallel_hook.py: the remaining trigger must match the graph backends' enable_memory_saver and SGLANG_MEMORY_SAVER_CUDA_GRAPH gate.
  • Behavior change on sglang-miles: DP-attention engines with a non-pausable pool regain NCCL graph registration.

🤖 Generated with Claude Code


CI States

Latest PR Test (Base): ❌ Run #36804707958
Latest PR Test (Extra): ❌ Run #36804707751
Latest PR Test (AMD ROCm 10): ❌ Run #36804707979

guapisolo and others added 2 commits September 30, 2026 19:09
Restore python/sglang/srt/arg_groups/parallel_hook.py and
test/registered/unit/server_args/test_server_args.py to their state
before #40977 cherry-picked #40648 at 0854ad8, which still carried the
DP-attention NCCL_GRAPH_REGISTER trigger. The next commit applies the
reviewed #40648 head instead. The other picks of #40977 are untouched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Apply the reviewed #40648 head: disable NCCL graph buffer registration
only when the CUDA graph pool is pausable (--enable-memory-saver with
SGLANG_MEMORY_SAVER_CUDA_GRAPH), and describe the hazard as stale
capture-time registrations after a torch_memory_saver resume
(fzyzcjy/torch_memory_saver#88). The DP-attention trigger is gone, so
DP-attention engines without a pausable pool keep NCCL's default
NCCL_GRAPH_REGISTER=1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@guapisolo
guapisolo merged commit 14a1fa7 into sglang-miles Oct 1, 2026
77 of 87 checks passed
@guapisolo
guapisolo deleted the jiajun/sglang-miles-nccl-pausable-only branch October 1, 2026 02:14
ziang-and pushed a commit to zianglih/sglang that referenced this pull request Oct 2, 2026
… graph registration (sgl-project#40648, sgl-project#40978, sgl-project#40979, sgl-project#40980, sgl-project#40977, sgl-project#41972)

Retain colocated training fixes above the release MiMo implementation, including address-stable Marlin reload and mandatory TopK layer IDs. Disable NCCL graph registration for TP LM-head all-to-all and pausable graph pools.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant