Skip to content

Fix dynamo recompile limit in allreduce and bf16 gemm - #32239

Merged
Qiaolin-Yu merged 3 commits into
mainfrom
fix_compile_limit
Jul 24, 2026
Merged

Fix dynamo recompile limit in allreduce and bf16 gemm#32239
Qiaolin-Yu merged 3 commits into
mainfrom
fix_compile_limit

Conversation

@Qiaolin-Yu

@Qiaolin-Yu Qiaolin-Yu commented Jul 23, 2026 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Collaborator

Motivation

Modifications

Accuracy Tests

Speed Tests and Profiling

Checklist

Review and Merge Process

  1. Ping Merge Oncalls to start the process. See the PR Merge Process.
  2. Get approvals from CODEOWNERS and other reviewers.
  3. Trigger CI tests with comments or contact authorized users to do so.
    • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
  4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

CI States

Latest PR Test (Base): 🚫 Run #30059916660
Latest PR Test (Extra): ❌ Run #30059916558

@gemini-code-assist

gemini-code-assist Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@Qiaolin-Yu

Qiaolin-Yu commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator Author

/rerun-test test/registered/cuda_graph/piecewise/test_pcg_glm52_fp4.py test/registered/quant/test_kimi_k26_nvfp4_dflash.py test/registered/8-gpu-models/test_kimi_k25.py

@github-actions

github-actions Bot commented Jul 23, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Results for /rerun-test test/registered/cuda_graph/piecewise/test_pcg_glm52_fp4.py test/registered/quant/test_kimi_k26_nvfp4_dflash.py test/registered/8-gpu-models/test_kimi_k25.py:

🚀 4-gpu-b200 (1 test): ❌ View workflow run 

cd test/ && python3 registered/cuda_graph/piecewise/test_pcg_glm52_fp4.py

🚀 8-gpu-b200 (2 tests): ❌ View workflow run 

cd test/ && python3 registered/quant/test_kimi_k26_nvfp4_dflash.py
cd test/ && python3 registered/8-gpu-models/test_kimi_k25.py

🚀 8-gpu-h200 (1 test): ✅ View workflow run 

cd test/ && python3 registered/8-gpu-models/test_kimi_k25.py

@Qiaolin-Yu

Qiaolin-Yu commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator Author

/rerun-test test/registered/cuda_graph/piecewise/test_pcg_glm52_fp4.py

@github-actions

github-actions Bot commented Jul 23, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Results for /rerun-test test/registered/cuda_graph/piecewise/test_pcg_glm52_fp4.py:

🚀 4-gpu-b200 (1 test): ❌ View workflow run 

cd test/ && python3 registered/cuda_graph/piecewise/test_pcg_glm52_fp4.py

@github-actions github-actions Bot added the quant LLM Quantization label Jul 23, 2026
@Qiaolin-Yu

Qiaolin-Yu commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator Author

/rerun-test test/registered/cuda_graph/piecewise/test_pcg_glm52_fp4.py test/registered/quant/test_kimi_k26_nvfp4_dflash.py test/registered/8-gpu-models/test_kimi_k25.py

@github-actions

github-actions Bot commented Jul 23, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Results for /rerun-test test/registered/cuda_graph/piecewise/test_pcg_glm52_fp4.py test/registered/quant/test_kimi_k26_nvfp4_dflash.py test/registered/8-gpu-models/test_kimi_k25.py:

🚀 4-gpu-b200 (1 test): ✅ View workflow run 

cd test/ && python3 registered/cuda_graph/piecewise/test_pcg_glm52_fp4.py

🚀 8-gpu-b200 (2 tests): ❌ View workflow run 

cd test/ && python3 registered/quant/test_kimi_k26_nvfp4_dflash.py
cd test/ && python3 registered/8-gpu-models/test_kimi_k25.py

🚀 8-gpu-h200 (1 test): ✅ View workflow run 

cd test/ && python3 registered/8-gpu-models/test_kimi_k25.py

@Qiaolin-Yu Qiaolin-Yu changed the title Fix dynamo recompile limit in allreduce Fix dynamo recompile limit in allreduce and bf16 gemm Jul 23, 2026
b8zhong
b8zhong reviewed Jul 23, 2026
View reviewed changes
Comment thread python/sglang/srt/distributed/parallel_state.py
if self.npu_communicator is not None and not self.npu_communicator.disabled:
return self.npu_communicator.all_reduce(input_)

if torch.compiler.is_compiling():

@b8zhong b8zhong Jul 23, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know if it will disable symmetric memory in BCG. Since it should be compatible there

@Qiaolin-Yu Qiaolin-Yu Jul 24, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think for bcg, torch.compiler.is_compiling() will be false

@b8zhong b8zhong Jul 24, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK

@b8zhong b8zhong added the run-ci label Jul 24, 2026
@Qiaolin-Yu

Qiaolin-Yu commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator Author

/rerun-test registered/quant/test_kimi_k26_nvfp4_dflash.py

@Qiaolin-Yu

Qiaolin-Yu commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator Author

/rerun-test registered/cuda_graph/piecewise/test_pcg_glm52_fp4.py

@github-actions

github-actions Bot commented Jul 24, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Results for /rerun-test registered/cuda_graph/piecewise/test_pcg_glm52_fp4.py:

🚀 4-gpu-b200 (1 test): ✅ View workflow run 

cd test/ && python3 registered/cuda_graph/piecewise/test_pcg_glm52_fp4.py

@github-actions

github-actions Bot commented Jul 24, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Results for /rerun-test registered/quant/test_kimi_k26_nvfp4_dflash.py:

🚀 8-gpu-b200 (1 test): ✅ View workflow run 

cd test/ && python3 registered/quant/test_kimi_k26_nvfp4_dflash.py

@Qiaolin-Yu
Qiaolin-Yu merged commit 15d73f1 into main Jul 24, 2026
115 of 151 checks passed
@Qiaolin-Yu
Qiaolin-Yu deleted the fix_compile_limit branch July 24, 2026 05:24
@Qiaolin-Yu Qiaolin-Yu mentioned this pull request Jul 24, 2026
Merged
Qiaolin-Yu added a commit that referenced this pull request Jul 24, 2026
@Qiaolin-Yu
[Cherry-pick to release/v0.5.16] Fix dynamo recompile limit in allred…
7505dc6 
…uce and bf16 gemm (#32239) (#32292)
@thanhhao98 thanhhao98 mentioned this pull request Jul 29, 2026
Closed
jinzhenfan pushed a commit to jinzhenfan/sglang that referenced this pull request Jul 29, 2026
@Qiaolin-Yu @jinzhenfan
Fix dynamo recompile limit in allreduce and bf16 gemm (sgl-project#32239
a0824f5 
)
Chronostasys pushed a commit to MindLab-Research/sglang that referenced this pull request Aug 24, 2026
@Qiaolin-Yu
[Cherry-pick to release/v0.5.16] Fix dynamo recompile limit in allred…
31e21f2 
…uce and bf16 gemm (sgl-project#32239) (sgl-project#32292)
jakki-amd pushed a commit to jakki-amd/sglang that referenced this pull request Sep 9, 2026
@Qiaolin-Yu @jakki-amd
Fix dynamo recompile limit in allreduce and bf16 gemm (sgl-project#32239
a7c178c 
)
Atituiset pushed a commit to Atituiset/sglang that referenced this pull request Sep 10, 2026
@Qiaolin-Yu
Fix dynamo recompile limit in allreduce and bf16 gemm (sgl-project#32239
3a5ba0b 
)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@b8zhong b8zhong b8zhong approved these changes
@merrymercy merrymercy Awaiting requested review from merrymercy merrymercy is a code owner
@Ying1123 Ying1123 Awaiting requested review from Ying1123 Ying1123 is a code owner
@Fridge003 Fridge003 Awaiting requested review from Fridge003 Fridge003 is a code owner
@ispobock ispobock Awaiting requested review from ispobock ispobock is a code owner
@HaiShaw HaiShaw Awaiting requested review from HaiShaw HaiShaw is a code owner
@ch-wan ch-wan Awaiting requested review from ch-wan ch-wan is a code owner
@BBuf BBuf Awaiting requested review from BBuf BBuf is a code owner
@Edwardf0t1 Edwardf0t1 Awaiting requested review from Edwardf0t1 Edwardf0t1 is a code owner
@yizhang2077 yizhang2077 Awaiting requested review from yizhang2077 yizhang2077 is a code owner
@FlamingoPg FlamingoPg Awaiting requested review from FlamingoPg FlamingoPg is a code owner
@AniZpZ AniZpZ Awaiting requested review from AniZpZ AniZpZ is a code owner
@OrangeRedeng OrangeRedeng Awaiting requested review from OrangeRedeng OrangeRedeng is a code owner
@Alisehen Alisehen Awaiting requested review from Alisehen Alisehen is a code owner

Assignees

No one assigned

Labels

quant LLM Quantization run-ci

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Qiaolin-Yu @b8zhong