Repository navigation
[sglang-miles] Sync #40648: drop the DP-attention NCCL graph registration trigger - #41972
Merged
Merged
Conversation
Restore python/sglang/srt/arg_groups/parallel_hook.py and test/registered/unit/server_args/test_server_args.py to their state before #40977 cherry-picked #40648 at 0854ad8, which still carried the DP-attention NCCL_GRAPH_REGISTER trigger. The next commit applies the reviewed #40648 head instead. The other picks of #40977 are untouched. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Apply the reviewed #40648 head: disable NCCL graph buffer registration only when the CUDA graph pool is pausable (--enable-memory-saver with SGLANG_MEMORY_SAVER_CUDA_GRAPH), and describe the hazard as stale capture-time registrations after a torch_memory_saver resume (fzyzcjy/torch_memory_saver#88). The DP-attention trigger is gone, so DP-attention engines without a pausable pool keep NCCL's default NCCL_GRAPH_REGISTER=1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ziang-and
pushed a commit
to zianglih/sglang
that referenced
this pull request
Oct 2, 2026
… graph registration (sgl-project#40648, sgl-project#40978, sgl-project#40979, sgl-project#40980, sgl-project#40977, sgl-project#41972) Retain colocated training fixes above the release MiMo implementation, including address-stable Marlin reload and mandatory TopK layer IDs. Disable NCCL graph registration for TP LM-head all-to-all and pausable graph pools. Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stop disabling NCCL graph registration for DP-attention engines whose graph pool is not pausable.
Symptom & Reproduction
--enable-dp-attention,--dp-size> 1 and decode CUDA graphs defaults toNCCL_GRAPH_REGISTER=0even without--enable-memory-saver, so its graph-captured collectives lose NCCL zero-copy registration.6e25d1bd57,test_without_all_to_all_env_is_untouchedresolves a DP-attention config (tp_size=2,dp_size=2, default decode graphs) and expectsNCCL_GRAPH_REGISTERto stay unset; it only holds because the test stubs_dp_attention_replays_decode_graphstoFalse. Without the stub,handle_data_parallelismsets it to0.Root Cause
#40977picked#40648at0854ad8c52, before review removed its DP-attention trigger._dp_attention_replays_decode_graphsfires for every DP-attention engine that replays decode graphs.#40648review found no hang from registration on a non-pausable pool, so upstream dropped it.Fix
Two commits, so the swap is reviewable:
616a8d299breverts only the fix(nccl): disable graph buffer registration for pausable graph pools #40648 hunks of [sglang-miles] Cherry-pick #40448, #40648, #40978, #40979, #40980 for MiMo-V2.6-Flash colocated RL #40977, restoringparallel_hook.pyandtest_server_args.pyto their pre-[sglang-miles] Cherry-pick #40448, #40648, #40978, #40979, #40980 for MiMo-V2.6-Flash colocated RL #40977 bytes; the other four picks are untouched.b35cf3e72capplies upstream fix(nccl): disable graph buffer registration for pausable graph pools #40648 at8739ae8063: only the pausable graph pool trigger (--enable-memory-saverwithSGLANG_MEMORY_SAVER_CUDA_GRAPH) remains, and the docs describe the hazard as stale capture-time registrations after a torch_memory_saver resume (NCCL_GRAPH_REGISTER=1 can hang after a same-VA TMS resume fzyzcjy/torch_memory_saver#88).Colocated miles engines (offload rollout plus miles' default
SGLANG_MEMORY_SAVER_CUDA_GRAPH=true) still getNCCL_GRAPH_REGISTER=0with no miles change; DP-attention engines without a pausable pool return to NCCL's default of1. An explicitNCCL_GRAPH_REGISTERstill wins, with a warning.Verification
git diff: after616a8d299bboth files equal sglang-miles before [sglang-miles] Cherry-pick #40448, #40648, #40978, #40979, #40980 for MiMo-V2.6-Flash colocated RL #40977; at this head the guard region ofparallel_hook.pyandTestTpLmHeadAllToAllNcclGraphRegisterare byte-identical to upstream fix(nccl): disable graph buffer registration for pausable graph pools #40648 at8739ae8063.ruff format --check,ruff check --select F, andpy_compileon the changed files: pass.test_server_args.py -k NcclGraphRegisterand GPU runs; the authoring host has no torch.Review Focus
_graph_pool_is_pausable/parallel_hook.py: the remaining trigger must match the graph backends'enable_memory_saver and SGLANG_MEMORY_SAVER_CUDA_GRAPHgate.🤖 Generated with Claude Code
CI States
Latest PR Test (Base): ❌ Run #36804707958
Latest PR Test (Extra): ❌ Run #36804707751
Latest PR Test (AMD ROCm 10): ❌ Run #36804707979