[alphabetc1]

Motivation

Eager request-window layout construction launches many small kernels to build attention indices, valid lengths, history metadata and commit masks. Fuse the pointwise work after the existing request-group scans to reduce launch overhead and temporary tensors.

Modifications

• Add two Triton kernels for query and history metadata on CUDA; retain the existing CPU and HIP implementation.
• Preserve padding, input strides, per-request floors, contiguous-run grouping and graph refresh behavior.
• Keep row counts and history size as runtime arguments to avoid compiling a separate kernel for each batch length.
• Keep the PR focused on the implementation; standalone UT and microbenchmark files were removed following review.

Accuracy Tests

Before the review cleanup, local validation on GB300 using test_dsv41_window_layout.py passed both tests: 64 shape/dtype/floor/padding combinations plus graph replay with changing request groups. Every output field is compared exactly with the CPU reference. The existing test_dsv41_request_window_copy.py passes all four tests. Repository pre-commit checks pass.

The standalone layout test was removed following review; the results above describe validation of the unchanged implementation at fc63b690ea. No full model accuracy score is claimed by this draft.

Speed Tests and Profiling

The benchmark used only generated request IDs and positions, with no model or dataset. Its script was removed from the final diff following review and remains available at commit fc63b690ea as benchmark/kernels/bench_dsv41_window_layout.py. To reproduce, extract that historical script and run it against each checkout:

git show fc63b690ea:benchmark/kernels/bench_dsv41_window_layout.py > /tmp/bench_dsv41_window_layout.py
PYTHONPATH=<checkout>/python python /tmp/bench_dsv41_window_layout.py

GB300, PyTorch 2.14.1+cu130 and Triton 3.8.0, eager wall time after warmup, 100 calls per sample and five samples per checkout. Input positions start at 8192; the benchmark fixes the window to 128 and supplies a floor tensor. Baseline is 780a5b1accd; candidate is fc63b690ea. Measurements are medians:

Rows / groups	Baseline (ms/call)	Fused (ms/call)
4096 / 1	0.897	0.407
32768 / 8	0.884	0.415

These are metadata-construction timings, not end-to-end serving speedups. Baseline samples ranged from 0.874–0.905 ms/call (4096 rows) and 0.873–0.905 ms/call (32768 rows); fused samples ranged from 0.397–0.444 and 0.414–0.452 ms/call, respectively.

Checklist

• Format code with repository pre-commit checks.
• Validate CPU-reference parity and graph replay on GB300.
• Record metadata-construction benchmark results.
• Remove standalone UT and benchmark files as requested in review.
• Full model accuracy and serving benchmark qualification.

---

CI States

Latest PR Test (Base): ❌ Run #39738861962
Latest PR Test (Extra): ❌ Run #39738861835
Latest PR Test (AMD ROCm 10): ❌ Run #39738861839

[alphabetc1]

/rerun-test test_deepseek_v4.py test_encoder_swa_replay.py test_dsv4_compressed_pools.py test_deepseek_v41_flash_gb300_encoder_bcg.py test_deepseek_v41_flash_gb300_dp4ep4.py

[github-actions]

🚀 4-gpu-b200 (1 test): ✅ View workflow run

cd test/ && python3 registered/attention/unittests/dsv4/test_deepseek_v4.py

🚀 1-gpu-h100 (1 test): ✅ View workflow run

cd test/ && python3 registered/attention/unittests/dsv4/test_deepseek_v4.py

🚀 ubuntu-latest (2 tests): ✅ View workflow run

cd test/ && python3 registered/unit/model_executor/test_encoder_swa_replay.py
cd test/ && python3 registered/unit/mem_cache/test_dsv4_compressed_pools.py

🚀 2-gpu-gb300 (1 test): ✅ View workflow run

cd test/ && python3 registered/e2e/models/test_deepseek_v41_flash_gb300_encoder_bcg.py

⛔ test_deepseek_v41_flash_gb300_dp4ep4.py: No test file found matching test_deepseek_v41_flash_gb300_dp4ep4.py under test/registered/ or python/sglang/multimodal_gen/test/.

[alphabetc1]

/rerun-test test_dsv41_request_window_copy.py test_deepseek_v41_flash_gb300_tp2ep2.py test_deepseek_v41_flash_gb300_dp2ep2.py test_deepseek_v41_flash_gb300_encoder_bcg.py test_deepseek_v41_flash_gb300_trtllm_bounded_bcg.py

[github-actions]

🚀 1-gpu-h100 (1 test): ✅ View workflow run

cd test/ && python3 registered/kernels/ops/kvcache/test_dsv41_request_window_copy.py

🚀 2-gpu-gb300 (4 tests): ✅ View workflow run

cd test/ && python3 registered/e2e/models/test_deepseek_v41_flash_gb300_tp2ep2.py
cd test/ && python3 registered/e2e/models/test_deepseek_v41_flash_gb300_dp2ep2.py
cd test/ && python3 registered/e2e/models/test_deepseek_v41_flash_gb300_encoder_bcg.py
cd test/ && python3 registered/e2e/models/test_deepseek_v41_flash_gb300_trtllm_bounded_bcg.py