[kevin-mii]

Motivation

#45407 made forward_low_ratio_sources cut x and q_lora to the hoisted rows on every step (x[:n], q_lora[:n]); before, it cut them only when DP attention padded an eager extend. On ROCm, q_lora (and on some routes x) reaches it quantized, as an Mxfp8Activation (q and scale) or an Fp8GridActivation: structs that refuse a slice. Since #45407, a DeepSeek-V4.1 server on MI35x fails while capturing its decode graphs (seen with --monokernel-backend amd; the failing call is SGLang's own forward, which every decode step of a plain server runs):

File ".../layers/attention/deepseek_v4_backend.py", line 3402, in forward_low_ratio_sources
    q_lora = q_lora[:n]
TypeError: 'Mxfp8Activation' object is not subscriptable

Modifications

• forward_low_ratio_sources cuts x, positions and q_lora only when DP padding left rows past the hoisted ones, as before [Fix] [NVIDIA] deepseek-v4.1: Fix the breakable prefill CUDA graph in the late-layer tail on SM90 #45407. Decode and verify steps, where every row is live, pass them on uncut.
• That cut goes through _first_rows, which cuts a quantized struct field by field (msgspec.structs.replace) and a tensor with [:n]. On CUDA both are tensors here, so CUDA takes the same cut as on main.

Accuracy Tests

MI350X, TP4, DeepSeek-V4.1-Flash, this branch on main 74c08f3, with sgl-project/sgl-amd-mono main 52dceb0:

• test/registered/unit/layers/test_dsv41_late_layer_tail_readers.py: 13 passed. The new case (DP padding cuts a quantized q_lora field by field) fails on main with the error above.
• test/registered/kernels/ops/attention/dsv4/test_dsv41_decode_glue_hip.py: 6 passed.
• sgl-amd-mono's test/e2e/test_deepseek_v41_server.py (--monokernel-backend amd): 9 passed. GSM8K: 0.930 (DSpark verify server), 0.905 (decode server). Both servers step past the monokernels' widths onto this path.
• SGLang alone, without the monokernels, the same two servers, GSM8K (200 questions, 8 shots): 0.895 (DSpark verify), 0.905 (decode).
• sgl-amd-mono's test/kernels/test_deepseek_v41_kernels.py against this SGLang (K1, K2 and the FFN launch bit for bit, with and without aiter's preshuffle, every width the spec builds): 6 passed, on 048555d.

The other GPU runs used e62ad22. The commits after it change only docstrings and the unit test.

Speed Tests and Profiling

No speed change intended: decode and verify steps now skip a cut that kept every row. AgentX (InferenceX agentic coding, TP4, concurrency 1, 900 s) with the monokernels, on this PR's first version (048555d, same decode and verify path): P90 637.6 tok/s/user, P50 668.0, 6627 tok/s/GPU, acceptance length 3.52.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.

🤖 Generated with Claude Code

---

CI States

Latest PR Test (Base): ⏳ Run #40030122492
Latest PR Test (Extra): ❌ Run #40030122499
Latest PR Test (AMD ROCm 10): ❌ Run #40030122510

[kevin-mii]

/tag-and-rerun-ci

[kevin-mii]

/rerun-test test/registered/e2e/models/test_deepseek_v41_flash_gb300_dp2ep2.py test/registered/e2e/models/test_deepseek_v41_flash_gb300_trtllm_bounded_bcg.py

[github-actions]

🚀 2-gpu-gb300 (2 tests): ✅ View workflow run

cd test/ && python3 registered/e2e/models/test_deepseek_v41_flash_gb300_dp2ep2.py
cd test/ && python3 registered/e2e/models/test_deepseek_v41_flash_gb300_trtllm_bounded_bcg.py