[GG] fix(ds4): preserve compressed MLA page stride

[voipmonitor]

Summary

Export the DeepSeek-V4 compressed MLA cache as its logical payload view while preserving the allocator's physical page stride.

Companion SparkInfer PR: local-inference-lab/b12x#106

Root cause

A 64-token DeepSeek-V4 page contains 37,376 payload bytes (64 * 584). The existing adapter unconditionally exposed the 37,440-byte SGLang-padded width. The 0731 checkpoint can instead receive a valid contiguous vLLM allocation whose physical page stride ends at 37,376 bytes, so forcing the padded view rejects or exceeds that allocation.

Implementation

• expose [pages, page_size * 584] as the logical view
• preserve stride(0) for packed/padded allocations
• accept contiguous exact-payload allocations
• reject short or overlapping physical page strides
• perform no copy and allocate no additional cache memory

Validation

• tests/models/deepseek_v4/test_b12x_cache_page_view.py: 3 passed
• paired SparkInfer test file: 12 passed
• TP2 B12X A8 MTP0: 145.7 tok/s sustained, 147.4 coding median
• TP2 B12X A8 DSpark K7: startup, KV initialization, CUDA graphs, and decode pass
• TP2 DSpark dynamic varlen path also passes

Summary by CodeRabbit

• Bug Fixes
  • Improved cache page handling for contiguous payloads and packed allocations.
  • Preserved physical page strides while exposing logical payload sizes.
  • Added validation to reject physical strides smaller than the required payload.
• Tests
  • Added coverage for contiguous pages, padded strides, and invalid stride configurations.

[coderabbitai]

Warning

Review limit reached

@voipmonitor, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 45 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 8a911981-ff89-4d42-a13f-651ba2b8bda5

📥 Commits

Reviewing files that changed from the base of the PR and between 78bc071 and fbc0226.

📒 Files selected for processing (2)

• tests/models/deepseek_v4/test_b12x_cache_page_view.py
• vllm/models/deepseek_v4/nvidia/b12x.py

📝 Walkthrough

Walkthrough

The B12x cache page view now exposes the exact logical payload, validates contiguous payload rows, preserves physical strides for packed allocations, and rejects strides smaller than the payload. Tests cover contiguous, padded, and invalid storage layouts.

Changes

B12x cache page payload and stride handling

Layer / File(s)	Summary
Logical payload sizing contract vllm/models/deepseek_v4/nvidia/b12x.py	Cache page sizing now uses the exact page_size * 584 payload size. Documentation describes contiguous and padded cache allocations.
Cache view stride handling vllm/models/deepseek_v4/nvidia/b12x.py, tests/models/deepseek_v4/test_b12x_cache_page_view.py	Cache views validate payload and physical strides, return logical payload slices, preserve packed-block strides, and test contiguous, padded, and invalid layouts.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

• local-inference-lab/vllm#47: Introduces the _b12x_cache_page_view packed KV cache handling refined by this change.

Suggested reviewers: lukealonso

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the DeepSeek-V4 compressed MLA page-stride preservation fix, which matches the main change.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/gg-ds4-compressed-page-view-20260731

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

vllm/models/deepseek_v4/nvidia/b12x.py (1)

82-116: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Validate page payload strides before exposing cached KV pages.

The 2D branch accepts page stride stride(0) < page_nbytes, which lets page rows overlap. The torch.as_strided branch accepts non-contiguous intra-page layouts, so the 1D payload can include bytes outside the logical token rows. Require stride(0) >= page_nbytes for every rank and add checks/tests for a 2D overlapping layout and a multidimensional gapped layout.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@vllm/models/deepseek_v4/nvidia/b12x.py` around lines 82 - 116, Update the
cache validation around the 2D and multidimensional branches to require page
stride(0) to be at least page_nbytes, rejecting overlapping rows for 2D tensors.
Before creating the as_strided page_view, also validate that the source layout
is contiguous within each page (stride(1) == 1) so gapped multidimensional
layouts cannot expose bytes outside the logical payload; preserve the existing
valid packed-block stride behavior and add coverage for both invalid layouts.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/models/deepseek_v4/test_b12x_cache_page_view.py`:
- Line 46: Update the pytest.raises match pattern in the relevant test to use a
raw string literal, changing the existing "page stride .* is smaller" pattern
while preserving the same regex and exception assertion.

In `@vllm/models/deepseek_v4/nvidia/b12x.py`:
- Around line 57-64: In vllm/models/deepseek_v4/nvidia/b12x.py at lines 57-64,
add Google-style `Args:` and `Returns:` sections to the docstring. The `Args:`
section should document the page_size parameter and the `Returns:` section
should describe the return value. At lines 72-76, add Google-style `Args:`,
`Returns:`, and `Raises:` sections, documenting the parameters, what the
function returns, and the validation failures mentioned in the existing
docstring text under the `Raises:` section. Follow the Google-style Python
docstring format with consistent formatting and type annotations.

---

Outside diff comments:
In `@vllm/models/deepseek_v4/nvidia/b12x.py`:
- Around line 82-116: Update the cache validation around the 2D and
multidimensional branches to require page stride(0) to be at least page_nbytes,
rejecting overlapping rows for 2D tensors. Before creating the as_strided
page_view, also validate that the source layout is contiguous within each page
(stride(1) == 1) so gapped multidimensional layouts cannot expose bytes outside
the logical payload; preserve the existing valid packed-block stride behavior
and add coverage for both invalid layouts.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 05671282-ec36-491c-9580-254a9fbfae96

📥 Commits

Reviewing files that changed from the base of the PR and between 3003860 and 78bc071.

📒 Files selected for processing (2)

• tests/models/deepseek_v4/test_b12x_cache_page_view.py
• vllm/models/deepseek_v4/nvidia/b12x.py

[voipmonitor]

Superseded for release integration by #229, which preserves this physical page-stride contract and combines it with the full compressed-MLA workspace-envelope fix from #227 on one clean GG head. The r21 candidate uses #229 only, so these overlapping changes are not applied twice.