[Refactor] Bind a layer stack in one pass over each stage's data flow

[ch-wan]

This PR is part of a stack (oldest at bottom):

• [Refactor] Declare every model's stages from a shared function and drop the meta-device neighbour build #43460
• [Refactor] Declare pipeline neighbours from a shared declaration function #43459
• [Refactor] Decide an exit's fixed facts when the stage is built #43458
• -> [Refactor] Bind a layer stack in one pass over each stage's data flow #43457
• [Refactor] Decide input-scattered eligibility and stage rejections in one place #43456
• [Refactor] Make residual read and update capabilities explicit and checked #43455
• [Fix] Keep residual rows and sums consistent across appends and pipeline handoffs #43454
• [Refactor] Record what a bound stage path does instead of reading it back #43453

Motivation

A stack bound its appends one at a time. The last stage of each append waited for the next append's first stage, and every connection between two appends was built twice (once as the waiting stage's outgoing connection, once as the next stage's incoming one, rebuilt by looking back through declaration history). Each stage was resolved twice per batch variant, and an attention's resolution left out the rows its residual is on while it runs, which each connection had to find elsewhere. That double resolution is where the two disagreeing rules fixed in #43454 came from.

Modifications

• Bind a layer stack's stages in one pass over its connections. The stack still binds when it closes, but chains its stages in append order and builds each connection once, as one stage's outgoing and the next one's incoming connection; a branch starts from the incoming connection of the stage it branches from. An attention that always leaves its sum binds after the FFN that follows it in its append, so its plan is built with that FFN's capture check and its consumer's input rows. Each stage records its producer without the producer's history.
• Resolve each stage's residual data flow once per batch variant. A stage is resolved once per variant, from what arrives at its entry and the stage after it, into one record: input and output contracts, the arrival, the rows the residual is on while the stage runs, and the rows its exit returns it on. Both edges of a stage are read from that record.
• Leave out what a stage's entry does not read of its producer's output. The arrival no longer copies the producer's sum group and carry permission when nothing is owed.

Accuracy Tests

How the stack is checked:

• Unit tests: the 39 layer-boundary test files pass at the head of every PR in the stack, apart from the two AMD-only test_aiter_allreduce_fusion_amd cases that fail on NVIDIA on main too; the test files that changed most recently pass on every commit. The full test/registered/unit suite (926 files, one process per file) shows no failure on the stack's head that main does not have.
• Plan comparison: an out-of-tree harness binds every stage plan for seven stack shapes (decoder layers with and without an output transform, MHC, Nemotron-H patterns, Kimi K3, Qwen4-exp, NemotronNAS-style attention-only layers, single-stage appends) over 23 parallel configurations, four boundary reductions and one or two pipeline ranks (3,928 scenarios), and compares each bound plan leaf by leaf with the parent commit's.
• GPU: dummy weights are seeded per TP rank and scaled (0.05, norms 1.0), so that a sum taken twice or a mixture of rank partials changes the output; the default dummy weights give every rank the same shard and would hide both.
• GPU results below were measured on these commits as based on main 516fd1c1b5 unless marked as on the merge candidate; rebasing onto current main had no conflicts, and the CPU checks were run again after it.
• On the merge candidate (head of this stack against main 961bcf481d, H200, asymmetric dummy weights unless noted): DeepSeek-V2-Lite input-scattered PP2 is bit-identical to PP1 (main crashes on it); Nemotron-H cut between a mixer and its FFN differs from PP1 only at the rounding level of the FFN-to-mixer cuts (main: 0.93 and 2.21); Llama, Qwen3-MoE, GPT-OSS and DeepSeek-V2-Lite are bit-identical to main at PP1 and PP2; GSM8K (real DeepSeek-V2-Lite weights, 250 questions, TP4 with attention DP2 and the dense FFN fully data-parallel) gives 0.360 at PP2, 0.364 at PP1 and 0.368 on main at PP1, medians of three runs whose run-to-run spread is 0.012–0.016. The last revision of the stack changed only comments, docstrings, an equivalent condition and explicitly set attributes after these runs; its bound plans are identical.

For this PR: the plan comparison shows no difference. Where a configuration is rejected for more than one reason, the message may now name another of them (all stages resolve before any binds); which scenarios are rejected does not change. The third commit differs only in arrival fields of arrivals that owe nothing, which no entry reads. Counting resolutions per stage per variant: 3.75 (decoder layers) and 6.62 (single-stage appends) on main, 1.00 here.

Speed Tests and Profiling

Construction resolves each stage once per variant instead of several times; no forward-path change.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

---

CI States

Latest PR Test (Base): 🚫 Run #38080821606
Latest PR Test (Extra): 🚫 Run #38080821429
Latest PR Test (AMD ROCm 10): ❌ Run #38080821654

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated
sglang-doc	🟢 Ready	View Preview	Oct 10, 2026, 7:44 PM

💡 Tip: Enable Automations to automatically generate PRs for you.