[ormandj]

Motivation

With SGLANG_RAGGED_VERIFY_MODE=compact and SGLANG_TEST_RAGGED_VERIFY_FORCE_UNIFORM_CAPTURE=1, DecodeCudaGraphRunner captures each token tier T with T // width full-width requests and no RaggedVerifyLayout. compute_target_verify_graph_key then keys the DeepSeek-V4 backend's captured metadata by request count, T // width. Every compact replay carries a layout and is keyed by token count, T.

A replay can select metadata captured for a different tier, or find no entry for its tier. Example: width 6, tiers 6/12/24/36. A 6-token replay finds key 6, which holds the metadata captured for the 36-token tier (6 requests). Copying the 6-token replay's metadata into buffers captured for 36 tokens fails on a shape mismatch; in my DeepSeek-V4.1-Flash DSpark runs this happened on the first replay.

Modifications

In compact mode with SGLANG_TEST_RAGGED_VERIFY_FORCE_UNIFORM_CAPTURE=1, DecodeCudaGraphRunner now captures each token tier with a RaggedVerifyLayout of tier // width full-width rows instead of no layout. The capture keeps the uniform geometry the flag asks for, but it records the packed attention path that replay refreshes, and it keys by token count like every compact capture, so compute_target_verify_graph_key is unchanged. A layout-free capture recorded expand_extend_with_same_length for the DeepSeek-V4 attention metadata. A packed replay that only matched its key would reuse those kernels: with two requests of 3 and 4 tokens in the 12-token tier, request 1's first three tokens (packed indices 3 to 5) would attend with request 0's KV and causal lengths. Other modes are unchanged.

This change covers the DeepSeek-V4 attention metadata. On DeepSeek-V4.1, compact replay, with or without this flag, also needs #42718, which maps packed verify tokens to their requests for the Engram hasher and the ratio-2 compressor, and #42709, which shares one capture layout between the two attention variants of a tier.

Accuracy Tests

TestForcedUniformCapture (CPU, test/registered/spec/dspark/test_ragged_verify.py) checks three things under forced-uniform compact capture: every tier's capture carries a full-width layout registered for replay staging, the key of a one-request replay padded to each tier finds that tier's capture, and staging two requests of 3 and 4 tokens into the 12-token tier moves the second request's start from token 6 to token 3. It fails on main, where the capture has no layout. test_ragged_verify.py, test_decode_cuda_graph_runner.py and test_dspark_scheduler.py: 45 passed on CPU.

Speed Tests and Profiling

Not applicable: test-only capture mode.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review and Merge Process

1. Ping Merge Oncalls to start the process. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

Developed with AI assistance.

---

CI States

Latest PR Test (Base): ❌ Run #40960545799
Latest PR Test (Extra): ❌ Run #40960545808
Latest PR Test (AMD ROCm 10): ❌ Run #40960545823