[SouthWest7]

Motivation

Fixes #45481. Deterministic hybrid inference can change greedy outputs when concurrent arrivals split a recurrent prefill chunk, or when a later request restores a prefix whose SSM state and attention KV were computed through different numerical paths.

Modifications

• Compose attention and SSM chunk alignment with LCM, and apply one intermediate-chunk rule to normal admission, exact chunk fill, ignore_eos, and continuing requests. Defer an intermediate cut that rounds to zero; allow a short final chunk. Preserve upstream DCP checkpoint alignment.
• Connect Triton GDN to the existing selected FP32 state snapshot support. Eager and prefill CUDA-graph tracking now cache the accumulator directly, avoiding a BF16 h round trip. Attention output still uses its existing activation-dtype h buffer.
• Under deterministic inference, the extra-buffer cache selects finished checkpoints within the original prompt. Decode-generated KV can differ numerically from prefill KV; retaining it during radix deduplication could contaminate a later prefill even when its restored SSM snapshot is exact. Prompt-prefix reuse remains enabled.

Rebased onto upstream b702f0d80e.

Accuracy Tests

Validated on H100 PCIe, BF16 models, Triton attention/linear attention, with deterministic inference enabled.

• Qwen3.5-9B: 32 prompts reconstructed using the concurrency recipe and exact token input attached to [Bug] Qwen3.5-9B deterministic greedy output differs with mixed-chunk scheduling #46446; 64 output tokens each. With radix cache enabled, concurrency 1, 8, 1, 8 produced 128/128 outputs identical token-for-token to upstream's cache-disabled, concurrency-1 reference. There were 36 actual prefix-cache hits, zero failures and zero retractions. Upstream produced 2/32 concurrency differences with radix off and 2/32 with radix on for this input set.
• Qwen3.5-35B-A3B, TP1/PP2 + NEXTN, step=3: the original thread/API-batch workload and forced arrival workload produced 63/63 valid 64-token responses. Each of the six inputs had exactly one output sequence across warmups and measured requests, including the 896/960-token cache-hit cases. All 63 responses recorded speculative verification; zero failures/retractions.
• Local CPU validation: 125 tests and 246 subtests passed, covering admission paths, alignment composition, cache checkpoint selection and tracking metadata. Development-only probes are retained locally and are not included in the commits.
• Local GPU validation: 13 existing GDN tests plus 2 local prefill-graph/checkpoint checks passed (36 subtests). The local checks verified bitwise-identical continuation outputs and final states after restoring a selected FP32 checkpoint, and CUDA-graph replay matching eager tracking across varied shapes and FP32/BF16 state pools. No new tests are included in the commits; the only test-file change adapts an existing test double to the scheduler's updated call signature.
• Repository-selected Ruff checks, formatting, isort and diff whitespace checks passed.

Serving comparisons used CUDA graphs disabled and mixed chunk disabled; prefill graph correctness was checked separately by capture/replay tests. #46525 remains complementary for deterministic mixed-batch decode tails.

Speed Tests and Profiling

No throughput benchmark or performance improvement claimed. Selected FP32 snapshots add one state buffer per sequence requiring intermediate tracking. Deterministic extra-buffer caching intentionally forgoes decode-generated prefix reuse to keep reusable KV and SSM checkpoints on the prefill numerical path.

---

CI States

Latest PR Test (Base): ❌ Run #41914992091
Latest PR Test (Extra): ❌ Run #41914991583
Latest PR Test (AMD ROCm 10): ❌ Run #41914991575

Tip

Base CI only covers the core cases, and its workload may not exercise your change. Use /rerun-test to run the tests your change touches, or copy this prompt for an agent:

Find the registered tests this PR affects and run them with /rerun-test, following https://github.com/sgl-project/sglang/blob/main/.agents/skills/rerun-test/SKILL.md

[MarinaMackay]

@SouthWest7 FYI, I opened #46525 for #46446, and the two PRs are complementary rather than overlapping. #46525 only changes the GDN backend: under deterministic inference it runs the decode tail of a mixed batch through the decode kernel. The issue's two-request repro needs that, and this PR does not touch it. This PR's chunk alignment covers the concurrency differences that remain with mixed chunk off.

I applied this PR on top of #46525 on 1x H200 (main d7f832d30a, Qwen3.5-9B, Triton, deterministic). Client concurrency 1 vs 8, 32 prompts, number of differing outputs:

	mixed off, radix off	mixed on, radix off	mixed on, radix on
#46525 only	3/32	2/32	6/32
#46525 + this PR	0/32	0/32	1/32

Also, this PR now conflicts with 8252e3af63 (#41701) in schedule_policy.py, which wraps both cut sites in floor_chunked_tokens. For the runs above I kept this PR's side of both hunks, which is equivalent without DCP.