Skip to content

[II] Warm Kimi B12X MLA DCP transport - #379

Closed
voipmonitor wants to merge 1 commit into
agent/ii-kimi-b12x-native-dcpfrom
agent/ii-kimi-b12x-dcp-warmup
Closed

voipmonitor wants to merge 1 commit into
agent/ii-kimi-b12x-native-dcpfrom
agent/ii-kimi-b12x-dcp-warmup

Conversation

@voipmonitor

Copy link
Copy Markdown

Status

Implemented. This pull request is stacked on #360.

Behavior

Kernel warmup recognizes Kimi-K3 attention layers that selected the B12X dense-MLA backend and initializes their decode-context-parallel query-gather and output-reduction transport signature before CUDA graph capture.

For Kimi-K3 TP16/DCP16, six local query heads become the 96-head DCP transport geometry, with a 576-element query head and a 512-element reduced output head. Duplicate module references share one warmed signature.

Technical reason

Kimi-K3 invokes its native attention adapter directly rather than through the generic MLAAttention module class. The existing B12X DCP warmup therefore did not discover the Kimi layer, leaving its transport specialization to first execution or graph capture.

Compatibility

DeepSeek-V4 and generic MLA discovery are unchanged. Kimi layers using another attention backend or DCP1 are ignored. The change allocates no persistent model storage beyond the transport resources already required by B12X DCP execution.

Validation

  • The generic MLA warmup regression test passes.
  • A focused Kimi-K3 test verifies one TP16/DCP16 signature with 96 total heads, 576 query dimensions, and 512 output dimensions.
  • Ruff check, Ruff format check, Python compilation, and git diff --check pass.
  • Both focused tests pass in the CUDA 13.3 / PyTorch 2.13 source-locked Kimi-K3 runtime image.

Composed serving qualification is tracked in local-inference-lab/rtx6kpro#66.

@coderabbitai

coderabbitai Bot commented Aug 15, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (1)
  • dev/*

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 913ffadb-1d3f-4ffa-b67e-ee5fc662e58a

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@voipmonitor

Copy link
Copy Markdown
Author

The behavior implemented by this pull request is preserved unchanged in vLLM #387 as commit 34fa68a87396. Stable Git patch IDs match. Review and merge #387; this pull request is closed to avoid duplicate review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant