Skip to content

[II] Select backend-owned Kimi dense MLA DCP - #360

Closed
voipmonitor wants to merge 1 commit into
agent/ii-b12x-mla-dcp-executionfrom
agent/ii-kimi-b12x-native-dcp
Closed

voipmonitor wants to merge 1 commit into
agent/ii-b12x-mla-dcp-executionfrom
agent/ii-kimi-b12x-native-dcp

Conversation

@voipmonitor

Copy link
Copy Markdown

Behavior

Kimi-K3 detects attention implementations that declare owns_decode_dcp_collectives=True. For those implementations, the model passes rank-local queries directly to the backend and accepts the backend's combined output. It does not allocate or invoke MLADCPManager for decode query gathering or output combination.

B12xMLAImpl declares that contract. B12xMLABackend accepts supported DCP geometries when prefill context parallelism is disabled, the gathered head count satisfies the eight-head kernel tile, and the largest local interleaved KV shard does not exceed 1,048,576 tokens. Kimi RoPE decode is permitted because the B12X backend gathers already-rotated rank-local queries.

Backends without the declaration retain the existing MLADCPManager path.

Dependencies

Depends on #359. TP16 serving also requires #338 for world-size-16 B12X transport, #339 for FP32 LSE transport, and #340 for bounded transport fallback.

Validation

Status: implemented and unit-qualified; composed serving qualification is pending.

  • ruff format, ruff check, and Python compilation: pass.
  • tests/v1/attention/test_b12x_mla.py plus tests/models/kimi_k3/test_sequence_parallel.py: 47 passed in the Kimi-K3 CUDA 13.3 / PyTorch 2.13 source-locked runtime image.
  • Selector cases cover TP16/DCP8, TP16/DCP16, unsupported DCP2 head geometry, and rejected prefill context parallelism.
  • Model cases verify that DCP1 and undeclared backends retain generic orchestration while a declared DCP backend owns the collectives.

@coderabbitai

coderabbitai Bot commented Aug 15, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (1)
  • dev/*

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: e2361e42-6e6d-4aca-94c8-5c026ab11db3

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@voipmonitor

Copy link
Copy Markdown
Author

The behavior implemented by this pull request is preserved in vLLM #387 as commit 88576e82b58a. The stable patch ID differs because the backend-ownership helper follows the consolidated sharded Kimi projection class. The consolidated range-diff retains the pull request contract. Review and merge #387; this pull request is closed to avoid duplicate review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant