Repository navigation
[ROCm][Kimi-K3] Support MoRIIO consumers with specialized decode - #63031
Merged
shen-shanshan merged 8 commits intoOct 23, 2026
Merged
shen-shanshan merged 8 commits into
shen-shanshan merged 8 commits into
Conversation
Allow the built-in MoRIIO consumer in READ and WRITE modes. Reuse the existing transfer completion guarantees, and keep other connectors and producer-side specialized execution unsupported. Add configuration coverage and document the supported deployment. Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com>
YukioZzz
requested review from
AndreasKaratzas,
DarkLight1337,
dllehr-amd,
hongxiayang,
tjtanaa and
ywang96
as code owners
October 22, 2026 04:19
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
Contributor
|
Documentation preview: https://vllm--63031.org.readthedocs.build/en/63031/ |
mergify
Bot
added
documentation
6 of 11 tasks
Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com>
Assisted-by: OpenAI Codex Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com>
Comment on lines
+107
to
+110
| ( | ||
| not supports_kv_transfer(vllm_config.kv_transfer_config), | ||
| "KV transfer connectors other than a MoRIIO consumer", | ||
| ), |
Contributor
There was a problem hiding this comment.
When kv_transfer_config is None (which means kv transfer is not enabled), supports_kv_transfer will return true, which is misleading and unclear. Can you directly enumerate the rejected cases for the bad part?
Assisted-by: OpenAI Codex Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com>
shen-shanshan
approved these changes
Oct 22, 2026
Contributor
|
/ci run |
|
✅ Triggered Buildkite CI #96637 for commit |
Set the multiprocessing backend explicitly for configuration-only coverage on single-GPU CI workers. Assisted-by: OpenAI Codex Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com>
Keep CUDA K3 config construction from enabling breakable graphs for subsequent tests. Use monkeypatch to restore the original environment. Assisted-by: OpenAI Codex Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com>
shen-shanshan
added
the
ready
|
✅ @YukioZzz, CI is now available for this PR.
|
Contributor
|
/ci run |
|
✅ Triggered Buildkite CI #96779 for commit |
seungrokj
added a commit
to seungrokj/vllm
that referenced
this pull request
Oct 26, 2026
Resolve the test_runtime.py conflict with vllm-project#63031 (MoRIIO consumers): keep the DCP 2 / 4 refusal cases, take the KVTransferConfig refusal case. Pin supported_vllm_config to a served DCP1 step (max_num_seqs 1, fp8 KV cache), which the (max_num_seqs, k) refusal now requires. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: seungrokj <seungrok.jung@amd.com>
bantianyue
pushed a commit
to bantianyue/vllm
that referenced
this pull request
Oct 28, 2026
…m-project#63031) Signed-off-by: Yichao Zhu <Yichao.Zhu@amd.com> Co-authored-by: Shanshan Shen <467638484@qq.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Enable Kimi-K3 specialized decode with MoRIIO READ/WRITE on the decode node, extending #60683 to P/D serving.
Claims
decode_backend=auto.Validation
48 and 40 tests passed. The four READ/WRITE × RDMA/xGMI acceptance cases fail without this change and pass with it.
Two-node MI355X, real Kimi-K3/DSpark weights, TP8/DCP8, K7, FULL decode graphs:
k3_mono_kda_pre_s8andk3_mono_k2_s8.This is a short functional check. xGMI has configuration coverage only. Two-node serving was tested locally because CI provides one eight-GPU node.
Details
Specialized layers skip per-layer transfer hooks. MoRIIO READ loads finish before forward (#60808); WRITE receives finish before scheduler admission. This PR enables that consumer path and leaves other connector configurations unchanged.
No overlapping open PR was found.
Pull Request Checklist
I used vLLM's
/pr-checklistskill. (Mandatory for agents, optional for humans).AI assistance was used during the creation of this PR.
Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.
Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.
Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.
Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.
BEFORE SUBMITTING, PLEASE READ https://docs.vllm.ai/en/latest/contributing