[Reshard] Plan Store KV reuse across TP, PP and host layouts by LLLL114 · Pull Request #4762 · kvcache-ai/Mooncake

LLLL114 · 2026-10-19T05:39:42Z

Description

First PR for #4719
Instances with different TP/PP configurations need to map stored KV data to their current placement. This PR adds a standalone Python planner that converts a source manifest and target placement into byte ranges covering every token of each page, ready for subsequent Store execution.

Define topology-independent logical page keys, model semantic domains, and immutable source manifests.
Plan reads through layer/head intersections across TP/PP configurations and PLHD, LPHD, and HPLD layouts.
Validate target memory bindings and lower logical plans into source Part offsets, destination offsets, and lengths, with range coalescing and batching.

Store execution, manifest publication/discovery, and framework integration follow in separate PRs.
The semantics for 'multipart' have not yet been fully implemented in Mooncake, so the review could be paused.

Suggested Merge Priority

P0 — Urgent: Critical security vulnerability, data loss/corruption, or widespread production outage requiring immediate action
P1 — High: Significant correctness or stability issue in existing functionality, or a regression blocking users without a practical workaround
P2 — Normal: Routine bug fix, validated performance/compatibility improvement, or new feature for a confirmed use case
P3 — Low: Experimental feature, optional enhancement, refactoring, or cleanup with no urgent user impact

Module

Type of Change

How Has This Been Tested?

Test commands:

PYTHONPATH=mooncake-reshard/python python -m pytest -q mooncake-reshard/tests
python -m pyright --project mooncake-reshard/pyrightconfig.json

Test results:

Unit tests pass
Integration tests pass (if applicable)
Manual testing done (describe below)

Checklist

I have performed a self-review of my own code
I have formatted my code using ./scripts/code_format.sh
I have run pre-commit on the files changed in this PR and all hooks pass
I have updated the documentation (if applicable)
I have added tests to prove my changes are effective
For changes >500 LOC: I have filed an RFC issue([RFC]: Mooncake Store KV Cache Reshard #4719)

AI Assistance Disclosure

No AI tools were used
AI tools were used (specify below)

Codex assisted with implementation, testing, and commit organization. The human submitter is responsible for final review.

Define semantic cache identities and immutable source layouts, compute target-driven logical intersections, and lower every token of each page into validated byte ranges. Plans identify source Part indices and relative offsets; Store RPCs, discovery, leases and data transfer are outside this commit. Include independent byte-exact layout and topology tests.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

Bo-Vincent · 2026-10-20T13:32:19Z

+    regions = {r.region_id: r for r in binding.regions}
+    groups: dict[tuple[int, KVCacheComponent], list[KVCacheResolvedRange]] = {}
+    spans: list[tuple[int, int]] = []
+    for item in binding.ranges:


Share the range geometry validation
validate_store_binding() duplicates much of the range validation in resolved.py. Could we extract a shared geometry validator while keeping Store and R2R identity checks separate? This would avoid maintaining the same bounds, stride, and coverage rules twice

Thanks for catching this. I extracted a shared geometry validator for bounds, strides, alignment, overlap, coverage, and validation limits. Store and R2R retain separate binding types and identity checks, including R2R snapshot validation.

Bo-Vincent · 2026-10-20T13:33:03Z

+        layout: KVCacheStoreLayout,
+        target_placement: KVCachePlacementManifest,
+        *,
+        runtime_format: KVCacheStoreFormat,


Remove the unused runtime_format parameter
runtime_format is only type-checked and does not affect the plan; destination layout comes from the runtime binding strides. Could we remove this parameter before the API is finalized?

Agreed. I removed runtime_format from the load-planning API and updated the callers and tests, including the page reader in the follow-up implementation.

… API Keep Store and R2R binding identity checks separate while sharing bounds, stride, coverage and validation-limit checks. Remove the unused runtime_format argument from Store load planning and update its tests. Signed-off-by: huanglong <huanglong@linux.alibaba.com>

Bo-Vincent

LGTM

…20261021 * upstream/main: (507 commits) [Reshard] Plan Store KV reuse across TP, PP and host layouts (kvcache-ai#4762) [Store] Walk tenants through shard-backed cursors and build the removes on them (kvcache-ai#4796) [Store-RS][Bugfix] Match TENT Request Layout to the C ABI (kvcache-ai#4732) [TransferEngine] Recover auto GID after data-path retry exhaustion (kvcache-ai#4290) [Store] Add the namespace layer over the object route (kvcache-ai#4721) [Store] Fix silent loss of offload keys deferred to the ungrouped pool (kvcache-ai#3006) (kvcache-ai#3009) [Store] Remove redundant alignment headroom from DFS shard allocation (kvcache-ai#4753) [CI/Build] Gate ROCm hardware tests and bound nightly Docker builds (kvcache-ai#4750) [Bugfix][Store] Fix double-counting of segment OpLog metrics in OpLogApplier (kvcache-ai#4771) [Store] Skip hot-cache writeback for device-resident read destinations (kvcache-ai#4764) [Reshard] Add bounded in-flight execution for weight transfers (kvcache-ai#4628) [Store] Avoid racing with detached hot-cache fills during key scans (kvcache-ai#4334) [Store] HA: drain the master on SIGTERM/SIGINT before exit (part 1 of kvcache-ai#4710) (kvcache-ai#4745) TENT: report a notification the receiver could not take (kvcache-ai#4754) [Store] Reach object metadata through a per-key accessor contract (kvcache-ai#4717) [Store][Bugfix] Fix DFS sync client test retry failure (kvcache-ai#4781) [Wheel] Keep integer dtype for typed_ragged rows when a row is an empty list (kvcache-ai#4611) [Doc] Sync README news onto the docs homepage (kvcache-ai#4767) [CI] remove stale Torch version binding after PG JIT (kvcache-ai#4760) [TransferEngine] Add buffer-protocol and page-gathered batch transfer APIs (kvcache-ai#4763) ...

…-ranges-20261021 * upstream/main: (394 commits) [Reshard] Plan Store KV reuse across TP, PP and host layouts (kvcache-ai#4762) [Store] Walk tenants through shard-backed cursors and build the removes on them (kvcache-ai#4796) [Store-RS][Bugfix] Match TENT Request Layout to the C ABI (kvcache-ai#4732) [TransferEngine] Recover auto GID after data-path retry exhaustion (kvcache-ai#4290) [Store] Add the namespace layer over the object route (kvcache-ai#4721) [Store] Fix silent loss of offload keys deferred to the ungrouped pool (kvcache-ai#3006) (kvcache-ai#3009) [Store] Remove redundant alignment headroom from DFS shard allocation (kvcache-ai#4753) [CI/Build] Gate ROCm hardware tests and bound nightly Docker builds (kvcache-ai#4750) [Bugfix][Store] Fix double-counting of segment OpLog metrics in OpLogApplier (kvcache-ai#4771) [Store] Skip hot-cache writeback for device-resident read destinations (kvcache-ai#4764) [Reshard] Add bounded in-flight execution for weight transfers (kvcache-ai#4628) [Store] Avoid racing with detached hot-cache fills during key scans (kvcache-ai#4334) [Store] HA: drain the master on SIGTERM/SIGINT before exit (part 1 of kvcache-ai#4710) (kvcache-ai#4745) TENT: report a notification the receiver could not take (kvcache-ai#4754) [Store] Reach object metadata through a per-key accessor contract (kvcache-ai#4717) [Store][Bugfix] Fix DFS sync client test retry failure (kvcache-ai#4781) [Wheel] Keep integer dtype for typed_ragged rows when a row is an empty list (kvcache-ai#4611) [Doc] Sync README news onto the docs homepage (kvcache-ai#4767) [CI] remove stale Torch version binding after PG JIT (kvcache-ai#4760) [TransferEngine] Add buffer-protocol and page-gathered batch transfer APIs (kvcache-ai#4763) ...

LLLL114 requested review from Bo-Vincent, ShangmingCai, stmatengss and zxpdemonio as code owners October 19, 2026 05:39

Copilot AI balanced review requested due to automatic review settings October 19, 2026 05:39

Copilot AI reviewed Oct 19, 2026

Merge branch 'main' into kv-reshard-pr1-planning

fe32cac

github-actions Bot added run-ci Common Tests labels Oct 19, 2026

[CI/Build] Fix ambiguous layer index in KV reshard tests

baf3d78

Bo-Vincent reviewed Oct 20, 2026

View reviewed changes

Bo-Vincent approved these changes Oct 21, 2026

View reviewed changes

Bo-Vincent merged commit e2d6e8e into kvcache-ai:main Oct 21, 2026
29 checks passed

kunkunblueberry mentioned this pull request Oct 22, 2026

[Reshard] Plan Store-backed KV cache reuse across host layouts #4741

Closed

7 of 36 tasks

ShangmingCai mentioned this pull request Oct 23, 2026

Bump version to 0.3.14 in pyproject.toml #4880

Merged

8 of 36 tasks

Navigation Menu

Repository navigation

[Reshard] Plan Store KV reuse across TP, PP and host layouts - #4762

[Reshard] Plan Store KV reuse across TP, PP and host layouts#4762
Bo-Vincent merged 4 commits into
kvcache-ai:mainfrom
LLLL114:kv-reshard-pr1-planning

LLLL114 commented Oct 19, 2026 •

edited by github-actions Bot

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Bo-Vincent Oct 20, 2026

Uh oh!

LLLL114 Oct 21, 2026

Uh oh!

Bo-Vincent Oct 20, 2026

Uh oh!

LLLL114 Oct 21, 2026

Uh oh!

Bo-Vincent left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

LLLL114 commented Oct 19, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Suggested Merge Priority

Module

Type of Change

How Has This Been Tested?

Checklist

AI Assistance Disclosure

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Bo-Vincent Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

LLLL114 Oct 21, 2026

Choose a reason for hiding this comment

Uh oh!

Bo-Vincent Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

LLLL114 Oct 21, 2026

Choose a reason for hiding this comment

Uh oh!

Bo-Vincent left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

LLLL114 commented Oct 19, 2026 •

edited by github-actions Bot

Loading