[II] Optimize DeepSeek V4 sparse metadata kernels

[voipmonitor]

Behavior

DeepSeek V4 sparse-attention metadata kernels specialize fixed strides, top-k width, and block size as Triton compile-time constants. The kernel that combines compressed top-k entries with the sliding-window region launches 256 workers per request instead of 128.

Output layout, integer arithmetic, and public interfaces are unchanged.

Technical reason

The global-index kernel receives fixed geometry for each compiled serving shape. Treating that geometry as runtime scalars prevents Triton from folding address calculations and loop bounds. The top-k/SWA kernel distributes long prefill rows across too few worker programs at 128 workers.

The implementation ports vLLM upstream PRs #51967 and #52084 to dev/infernal-invocation.

Compatibility

• DeepSeek V4 metadata values and tensor shapes are unchanged.
• CUDA graph capture uses the same persistent buffers and dispatch keys.
• Models outside the DeepSeek V4 sparse metadata path are unaffected.

Validation

SM120 A/B/A measurements used GPU 6 on an RTX PRO 6000 Blackwell Server Edition direct-attach host. Both variants ran in the same Infernal Invocation image with deterministic inputs, 100 warmup iterations, and 300 measured iterations. The comparator was dev/infernal-invocation@6dc2f516688f.

Kernel	Tokens	Comparator	Optimized	Latency change
Global top-k slot mapping	1	7.81 us	6.66 us	-14.7%
Global top-k slot mapping	32	8.36 us	7.55 us	-9.7%
Global top-k slot mapping	512	10.16 us	8.82 us	-13.2%
Top-k plus SWA combination	1,024	10.92 us	9.58 us	-12.2%
Top-k plus SWA combination	4,096	26.75 us	22.72 us	-15.1%
Top-k plus SWA combination	8,192	51.74 us	44.34 us	-14.3%
Top-k plus SWA combination	16,384	95.91 us	80.50 us	-16.1%

All output and length checksums matched across the A/B/A runs.

• pytest -q tests/models/deepseek_v4/test_cache_utils.py: 3 passed on SM120.
• The worker-tile test compares two requests and 300 query rows against a PyTorch reference, covering rows beyond one 256-worker tile.
• Ruff check, format validation, and git diff --check: passed.

[coderabbitai]

Warning

Review limit reached

@voipmonitor, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 26 minutes

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits within each organization.

For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d1b250bf-8e54-429d-b079-fbc5b8cb286d

📥 Commits

Reviewing files that changed from the base of the PR and between 6dc2f51 and 2822782.

📒 Files selected for processing (2)

• tests/models/deepseek_v4/test_cache_utils.py
• vllm/models/deepseek_v4/common/ops/cache_utils.py

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}