Skip to content

dcp: make head-sliced attention views contiguous for the B12X PCIe pool (fathomless-firmament) - #87

Merged
lukealonso merged 1 commit into
dev/fathomless-firmamentfrom
fable/dcp-b12x-contiguous-lse-ff-20260709
Jul 9, 2026
Merged

lukealonso merged 1 commit into
dev/fathomless-firmamentfrom
fable/dcp-b12x-contiguous-lse-ff-20260709

Conversation

@voipmonitor

Copy link
Copy Markdown

Same single commit as #81, retargeted at dev/fathomless-firmament (which already carries the #78/#79 re-picks but not this fix — _try_b12x_dcp_lse_reduce still passes head-sliced views straight to the pool).

Problem

Sparse MLA backends can return head-sliced views: GLM TP6 pads 64 → 66 attention heads and slices the kernel output back (lse = lse[:, :input_num_heads]), producing non-contiguous cp_attn_out/cp_attn_lse. The B12X PCIe DCP pool validates contiguity, so TP6 × DCP≥2 dies at CUDA-graph capture with ValueError: partial_lse must be contiguous. TP8 never hits it (kernel head count == input head count); the NCCL packers take explicit strides and are unaffected — only this fast path needs the copy.

Fix

Guarded .contiguous() on both operands in _try_b12x_dcp_lse_reduce, after the cheap reject checks and before the pool call. LSE is a tiny [B, H] fp32 tensor and the output copy only happens on padded head counts; capture-time allocations go to the graph pool like the existing send/recv buffers.

Verification

  • Unit test simulates the head66 slicing pattern (padded buffers sliced to [:, :16], asserts the pool receives contiguous tensors). Suite on the v6 image with this file mounted: 31 passed + 1 pre-existing environment-bound failure (TestDCPCommBackendConfig::test_a2a_with_dcp_valid, identical on the unpatched branch in the same container).
  • E2E on the eldritch lineage (v6 image, GLM-5.2 AMD MXFP4, TP6/A8/online-MXFP8/DCP2, GMU 0.95): previously died at capture; with the fix boots (KV 639,616), c0 67.1 / c3000 66.9 tok/s, 0 CJK.
  • Content-equivalent to the build patch blackwell-llm-docker/patches/vllm-dcp-b12x-contiguous-lse-20260707.patch — merging retires it.

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Jul 9, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@voipmonitor, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 29 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 1ecbb4be-4efa-4e61-be0d-1a765a342a80

📥 Commits

Reviewing files that changed from the base of the PR and between 407ace1 and 210a68a.

📒 Files selected for processing (2)
  • tests/distributed/test_dcp_a2a.py
  • vllm/v1/attention/ops/dcp_alltoall.py
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fable/dcp-b12x-contiguous-lse-ff-20260709

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@lukealonso
lukealonso force-pushed the dev/fathomless-firmament branch from 3a864ad to 407ace1 Compare July 9, 2026 23:24
Sparse MLA backends can return the decode output and LSE as head-sliced
views: GLM TP6 virtual-TP pads 64 -> 66 attention heads, the B12X sparse
kernel pads its head dim further and slices the result back
(b12x_mla_sparse lse = lse[:, :input_num_heads]), which is non-contiguous
whenever the kernel head count exceeds the input head count. The B12X
PCIe DCP pool validates contiguity, so TP6 + DCP>1 with the A2A fast
path died at CUDA graph capture with 'partial_lse must be contiguous'.
TP8 shapes never hit this because kernel and input head counts match.

Copy the operands to contiguous in _try_b12x_dcp_lse_reduce, after the
cheap reject checks so NCCL fallback paths pay nothing. The NCCL packers
take explicit strides and never needed this; the query all-gather wrapper
already does the same for its input. The LSE copy is [B, H] fp32 and the
output copy only happens on padded head counts.

Diagnosed by Martin on TP6/MXFP4/A8/DCP2 (v6 image); unit test simulates
the sliced-view pattern and asserts the pool receives contiguous tensors.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@voipmonitor
voipmonitor force-pushed the fable/dcp-b12x-contiguous-lse-ff-20260709 branch from b65f864 to 210a68a Compare July 9, 2026 23:27
@lukealonso
lukealonso merged commit c649d41 into dev/fathomless-firmament Jul 9, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants