[Refactor] Construct independent decoder stage boundaries

[ch-wan]

This PR is part of a stack (oldest at bottom):

• [Refactor] Document layer boundary contracts and integration #41557
• [Refactor] Group layer boundary unit tests #41556
• [Refactor] Rename the module to layer_boundary #41555
• [Refactor] Retire the layer facade and simplify boundary internals #41554
• [Refactor] Migrate specialized decoder and overlap boundaries #41553
• -> [Refactor] Construct independent decoder stage boundaries #41552
• [Refactor] Select reduction fusion at the consumer #41551
• [Refactor] Capture auxiliary states at residual reads #41550
• [Refactor] Carry residual state across stage boundaries #41549
• [Refactor] Centralize decoder output access #41548
• [Refactor] Group communicator fusion and CP adapters #41547

Motivation

A fixed attention/FFN facade obscures each stage's input, output, and reduction contract. Independent declarations allow ordinary decoders and longer stage sequences to use the same construction API.

Modifications

• Add attention/FFN declarations, independent stage factories, and make_stages(). Migrate ordinary decoders, their wrappers, PP and split-prefill callers together. Reuse the batch-owned residual lifecycle introduced by [Refactor] Carry residual state across stage boundaries #41549; model forwards now pass the hidden output without threading a second stream return value.
• At this revision, the reduction option is consumed only by decoders migrated to independent stage boundaries. Remaining decoder families adopt it in [Refactor] Migrate specialized decoder and overlap boundaries #41553.
• Add --boundary-reduction {auto,ar,rs,rsv,rs+rsv} (default auto) and the derived speculative_boundary_reduction server-args field. Resolve target and separate draft defaults before construction and document them in the server-argument reference.
• Permit deferred FFN completion for Qwen3 and its wrappers, Llama4, SDAR, LLaDA2 and Step3-VL. Without reduction fusion this moves the same all-reduce to the following prepare. With an enabled compatible backend it permits fused all-reduce+add+norm, with potentially different rounding. Capture runs on the main prepare path to avoid a second reduction.
• Use one model-wide reduction policy for dense and MoE layers. Under the rsv defaults, eligible dense Bailing linear/V2.5 and Step3-VL layers can switch from all-reduce plus scatter to reduce-scatterv. Dense Qwen3 backbones retain ar, including detected wrappers.
• Require an explicit boundary request before MoE skips its post-expert reduction; topology alone is no longer enough. Update DeepSeek-V4 and Qwen4-Exp to publish that request.
• Combine Step3-VL routed and shared partials before one reduction. This changes accumulation order and removes the duplicate reduction previously possible with DP/EP and no expert all-to-all.
• Pass the NPU weight-cache hint per preparation call instead of retaining it on the layer.

Accuracy Tests

1 focused CPU test-file runs passed on H200 against the current head (24 reported test cases). These cover the review changes; the previous revision also passed the broader boundary CPU suite. Collective tests use CPU stand-ins. No new GPU numerical or throughput benchmark was run.

Speed Tests and Profiling

The collective/fusion changes above are intentional. ar disables optional scatter; fusion backend settings control fused all-reduce. No new GPU benchmark was run.

Known follow-up: the earlier DP prefill-graph measurement was about 1.18 GiB versus 1.12 GiB on main. The revised stack does not claim memory parity for this path.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review and Merge Process

1. Ping Merge Oncalls to start the process. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

---

CI States

Latest PR Test (Base): 🚫 Run #36516447798
Latest PR Test (Extra): 🚫 Run #36516447959
Latest PR Test (AMD ROCm 10): 🚫 Run #36516447970

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated
sglang-doc	🟢 Ready	View Preview	Sep 29, 2026, 3:19 AM

💡 Tip: Enable Automations to automatically generate PRs for you.