[Refactor] Leave decoder stacks through final_norm(skip_empty=True)

[ch-wan]

This PR is part of a stack (oldest at bottom):

• [Refactor] Build seven more decoders from stage boundaries #41779
• [Refactor] Simplify layer boundary internals #41778
• [Fix] Step-3.5: build the shared expert without TP under all-to-all MoE backends #41754
• [Refactor] Send producer-written residual streams through to_pp #41753
• [Fix] Reduce the Sarvam dense MLP through its row-parallel projection #41752
• -> [Refactor] Leave decoder stacks through final_norm(skip_empty=True) #41751
• [Refactor] Rename layer boundary protocols, contracts and helpers #41750
• [Fix] Release consumed residual contributions #41749

Motivation

Twelve decoder stacks finished the last layer's output with residual_batch.complete_output() and then called final_norm() only for a non-empty batch. final_norm() already completes owed work before the norm, and skip_empty=True skips only the norm kernel on a zero-row output, so the pair did the same work as one call. On an empty batch the old form also left the batch's residual stream in place instead of releasing it. Four split-prefill paths called complete_output() right before an unconditional final_norm().

Modifications

• complete_output() followed by a non-empty-guarded final_norm() becomes final_norm(..., skip_empty=True) (GigaChat3.5, Intern-S2-Mobius, Laguna, MiniMax-M2, MiniMax-M3, Qwen2-MoE, Qwen3, Qwen3-VL, Qwen3-VL-MoE, Sarvam, Step3-VL, XLLM). Capture callbacks and post_residual_addition are passed as before and, as before, not applied to an empty batch.
• The redundant complete_output() before an unconditional final_norm() is removed (Qwen2-MoE, Qwen3-MoE, Sarvam ×2).
• Stacks that guard the norm with forward_mode.is_idle() are unchanged: an idle DP batch can carry padded rows, so that guard is not equivalent. Stacks that capture the completed output before the norm are also unchanged.

Accuracy Tests

B200, --tp-size 2 --dp-size 2 --enable-dp-attention:

• one_batch --correctness-test (prefill logits and three generations) is identical to the parent commit for Qwen3-0.6B, Qwen1.5-MoE-A2.7B and Qwen3-VL-2B-Instruct. one_batch gives every DP rank the same prompts, so no rank is empty there.
• To reach the empty-rank path, a server (--disable-radix-cache --random-seed 42) received four single greedy requests one at a time, each served by one DP rank while the other stayed idle. Text, input and output logprobs and the top-5 logprobs are identical to the parent commit for Qwen1.5-MoE-A2.7B and Qwen3-VL-2B-Instruct, and a repeat run of the parent is identical too.

H200, MiniMax-M3 cut to five layers (three dense, two MoE), sglang.Engine with --tp-size 2 --dp-size 2 --enable-dp-attention and single requests alternating between the DP ranks: the output and the logits of every step are bitwise identical to the parent commit on both ranks. On the idle rank's zero-row forwards this PR's stack calls final_norm(skip_empty=True), so the empty path is exercised.

97 affected unit test files (2,081 tests) pass.

Speed Tests and Profiling

Not applicable: the same kernels and collectives run.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review and Merge Process

1. Ping Merge Oncalls to start the process. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

---

CI States

Latest PR Test (Base): ⏳ Run #36667496735
Latest PR Test (Extra): ⏳ Run #36667496474
Latest PR Test (AMD ROCm 10): ⏳ Run #36667496688

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated
sglang-doc	🟡 Building	–	Sep 30, 2026, 4:05 AM

💡 Tip: Enable Automations to automatically generate PRs for you.