Skip to content

[Store] Allow DFS-only replica placement - #4637

Merged
ykwd merged 1 commit into
kvcache-ai:mainfrom
2xdevv:feat/dfs-only-placement
Oct 19, 2026
Merged

[Store] Allow DFS-only replica placement#4637
ykwd merged 1 commit into
kvcache-ai:mainfrom
2xdevv:feat/dfs-only-placement

Conversation

@2xdevv

@2xdevv 2xdevv commented Oct 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Description

Allow DFS-only placement with replica_num = 0, nof_replica_num = 0, and dfs_replica_num = 1. Previously, PutStart and UpsertStart rejected DFS requests without a memory replica.

Add lifecycle coverage and update the C++, Python, and deployment documentation. When DFS is the only copy, DFS eviction or object removal deletes the object. Existing same-size upsert topology checks and DFS tenant, recovery, and durability limitations remain unchanged.

Module

  • Mooncake Store (mooncake-store)
  • Docs

Type of Change

  • New feature
  • Documentation update

How Has This Been Tested?

Added tests covering:

  • DFS-only put/get, removal, re-put, same-size and resized upserts, and batch operations.
  • Per-key batch write failure isolation and upsert after memory replica loss.
  • Extent reclamation after revoke, write preemption, removal, and timeout.
  • Survival of memory eviction and segment loss, plus whole-bucket eviction and write failure cleanup.

Validation performed:

  • git diff HEAD^ HEAD --check: passed.
  • Changed-line formatting check against HEAD^: passed.
  • Pre-commit on the seven changed files: all applicable hooks passed after automatic whitespace correction. The correction affected a pre-existing line outside this change and was reverted; HEAD still triggers that trailing-whitespace check.
  • Runtime tests: all four test suites passed on a fresh Debug build using POSIX DFS and TCP. The main run passed 98 tests and skipped two opt-in checksum tests; both passed with checksums enabled in a follow-up run. All 100 unique tests passed, including all seven new tests.
  • Sphinx HTML build: passed with warnings treated as errors. The three changed pages and the DFS deployment cross-reference were verified in the generated HTML.

Test results:

  • Unit tests pass
  • Integration tests pass
  • Manual testing done

Checklist

  • I have performed a self-review of my own code
  • I have checked formatting using ./scripts/code_format.sh
  • I have run pre-commit on the files changed in this PR
  • I have updated the documentation
  • I have added tests to cover the change

AI Assistance Disclosure

  • AI tools were used

Codex assisted with code review, local validation, and drafting this PR description. The human submitter is responsible for reviewing every changed line and understanding the change end-to-end.

@github-actions github-actions Bot added documentation Improvements or additions to documentation run-ci Store labels Oct 13, 2026
fcczzz
fcczzz approved these changes Oct 14, 2026
View reviewed changes

@fcczzz fcczzz left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The existing allocation, transfer finalization, read, and cleanup paths support DFS-only replica lists. Removing the admission restriction is consistent with those paths, and the added tests cover the main lifecycle and failure cases. LGTM

@ykwd
ykwd merged commit c6f1d08 into kvcache-ai:main Oct 19, 2026
27 checks passed
@2xdevv
2xdevv deleted the feat/dfs-only-placement branch October 19, 2026 10:46
This was referenced Oct 19, 2026
Closed
Merged
Open
@2xdevv 2xdevv mentioned this pull request Oct 20, 2026
Merged
13 of 14 tasks
@Juhyun-Kim-Memphis Juhyun-Kim-Memphis mentioned this pull request Oct 20, 2026
Open
9 of 32 tasks
daegyu94 pushed a commit to daegyu94/vllm that referenced this pull request Oct 20, 2026
Daegyu Han
[KV Connector][Mooncake] Allow DFS-only replica placement
4248270 
Mooncake now accepts replica_num=0 with dfs_replica_num=1
(kvcache-ai/Mooncake#4637), storing the object only in DFS. Relax the
replica_num check from >= 1 to >= 0 and document the requirement.

Signed-off-by: Daegyu Han <daegyu94.han@samsung.com>
daegyu94 pushed a commit to daegyu94/vllm that referenced this pull request Oct 20, 2026
Daegyu Han @claude
[KV Connector] Enable DFS in MooncakeStoreConnector
5a711fb 
Add opt-in DFS support to MooncakeStoreConnector. With "enable_dfs": true
in the Mooncake JSON config, the connector passes enable_ssd_offload=True
to the native SDK, which creates FileStorage and its distributed backend,
and forwards replica_num and dfs_replica_num from kv_connector_extra_config
to ReplicateConfig. The existing vLLM enable_offload option still only
controls GET budgeting for external SSD owners.

replica_num may be 0 for DFS-only placement, which Mooncake supports since
kvcache-ai/Mooncake#4637; dfs_replica_num must be 1. Validate SDK support,
replica counts, and the default-tenant restriction before Store setup, and
reject DFS in the RAM-only cross-encoder cache. DFS reads reuse the GET
batch splitter with local_buffer_size as the per-call budget. Document the
Master and client settings, including MOONCAKE_OFFLOAD_FILE_STORAGE_PATH.

Validation: Mooncake connector unit tests (336) and pre-commit pass. A
native 3FS/CUDA run on Mooncake main d855a6ae with Qwen2.5-0.5B-Instruct
stored 33 keys from a 537-token prompt. With replica_num=0 none had a
memory replica (Master memory usage 0 B); with replica_num=1 all did.
After resetting the local prefix cache, both runs loaded 528 tokens from
the Store and the 32 output token IDs matched the first generation.

Aggregate buffer accounting across concurrent receivers is deferred;
multiple receivers can exhaust the shared buffer. Multi-node, RDMA, and
long-duration stress remain untested.

Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: Daegyu Han <daegyu94.han@samsung.com>
daegyu94 pushed a commit to daegyu94/vllm that referenced this pull request Oct 20, 2026
Daegyu Han @codex
[KV Connector] Enable DFS in MooncakeStoreConnector
404b565 
Add opt-in DFS support to MooncakeStoreConnector. With "enable_dfs": true
in the Mooncake JSON config, the connector passes enable_ssd_offload=True
to the native SDK, which creates FileStorage and its distributed backend,
and forwards replica_num and dfs_replica_num from kv_connector_extra_config
to ReplicateConfig. The existing vLLM enable_offload option still only
controls GET budgeting for external SSD owners.

replica_num may be 0 for DFS-only placement, which Mooncake supports since
kvcache-ai/Mooncake#4637; dfs_replica_num must be 1. Validate SDK support,
replica counts, and the default-tenant restriction before Store setup, and
reject DFS in the RAM-only cross-encoder cache. DFS reads reuse the GET
batch splitter with local_buffer_size as the per-call budget. Document the
Master and client settings, including MOONCAKE_OFFLOAD_FILE_STORAGE_PATH.

Validation: Mooncake connector unit tests (336) and pre-commit pass. A
native 3FS/CUDA run on Mooncake main d855a6ae with Qwen2.5-0.5B-Instruct
stored 33 keys from a 537-token prompt. With replica_num=0 none had a
memory replica (Master memory usage 0 B); with replica_num=1 all did.
After resetting the local prefix cache, both runs loaded 528 tokens from
the Store and the 32 output token IDs matched the first generation.

Aggregate buffer accounting across concurrent receivers is deferred;
multiple receivers can exhaust the shared buffer. Multi-node, RDMA, and
long-duration stress remain untested.

Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Daegyu Han <daegyu94.han@samsung.com>
daegyu94 pushed a commit to daegyu94/vllm that referenced this pull request Oct 20, 2026
Daegyu Han @claude
[KV Connector] Enable DFS in MooncakeStoreConnector
c6e13fd 
Add opt-in DFS support to MooncakeStoreConnector. With "enable_dfs": true
in the Mooncake JSON config, the connector passes enable_ssd_offload=True
to the native SDK, which creates FileStorage and its distributed backend,
and forwards replica_num and dfs_replica_num from kv_connector_extra_config
to ReplicateConfig. The existing vLLM enable_offload option still only
controls GET budgeting for external SSD owners.

replica_num may be 0 for DFS-only placement, which Mooncake supports since
kvcache-ai/Mooncake#4637; dfs_replica_num must be 1. Validate SDK support,
replica counts, and the default-tenant restriction before Store setup, and
reject DFS in the RAM-only cross-encoder cache. DFS reads reuse the GET
batch splitter with local_buffer_size as the per-call budget. Document the
Master and client settings, including MOONCAKE_OFFLOAD_FILE_STORAGE_PATH.

Validation: Mooncake connector unit tests (335) and pre-commit pass. A
native 3FS/CUDA run on Mooncake main d855a6ae with Qwen2.5-0.5B-Instruct
stored 33 keys from a 537-token prompt. With replica_num=0 none had a
memory replica (Master memory usage 0 B); with replica_num=1 all did.
After resetting the local prefix cache, both runs loaded 528 tokens from
the Store and the 32 output token IDs matched the first generation.

Aggregate buffer accounting across concurrent receivers is deferred;
multiple receivers can exhaust the shared buffer. Multi-node, RDMA, and
long-duration stress remain untested.

Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: Daegyu Han <daegyu94.han@samsung.com>
daegyu94 pushed a commit to daegyu94/vllm that referenced this pull request Oct 20, 2026
Daegyu Han @claude
[KV Connector][Mooncake] Add DFS support to MooncakeStoreConnector
4acfbd9 
Add an opt-in "enable_dfs" option to the Mooncake JSON config. When set,
the connector initializes Mooncake's DFS backend during Store setup and
writes one DFS replica per object plus replica_num memory replicas
(kv_connector_extra_config, default 1). replica_num=0 stores objects
only in DFS and requires kvcache-ai/Mooncake#4637.

DFS is limited to the default tenant and is rejected for the
cross-encoder cache. DFS loads are split so each GET fits in
local_buffer_size.

Tested with the Mooncake store unit tests and a 3FS run on Mooncake
main (d855a6ae) with replica_num 0 and 1.

Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: Daegyu Han <daegyu94.han@samsung.com>
@daegyu94 daegyu94 mentioned this pull request Oct 20, 2026
Open
6 tasks done
daegyu94 pushed a commit to daegyu94/vllm that referenced this pull request Oct 21, 2026
Daegyu Han @claude
[KV Connector][Mooncake] Add DFS support to MooncakeStoreConnector
80deda8 
Add an opt-in "enable_dfs" option to the Mooncake JSON config. When set,
the connector initializes Mooncake's DFS backend during Store setup and
writes one DFS replica per object plus replica_num memory replicas
(kv_connector_extra_config, default 1). replica_num=0 stores objects
only in DFS and requires kvcache-ai/Mooncake#4637.

DFS is limited to the default tenant and is rejected for the
cross-encoder cache. DFS loads are split so each GET fits in
local_buffer_size, or in the SSD owner's staging buffer if smaller when
enable_offload is also set, since Mooncake reads SSD replicas first.

Tested with the Mooncake store unit tests and a 3FS run on Mooncake
main (d855a6ae) with replica_num 0 and 1.

Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: Daegyu Han <daegyu94.han@samsung.com>
@wooyeonlee0 wooyeonlee0 mentioned this pull request Oct 21, 2026
Open
4 of 5 tasks
@daegyu94 daegyu94 mentioned this pull request Oct 30, 2026
Merged
9 tasks done
daegyu94 pushed a commit to daegyu94/vllm that referenced this pull request Oct 31, 2026
Daegyu Han @claude @daegyu94
[KV Connector][Mooncake] Add DFS support to MooncakeStoreConnector
f1368c5 
Add an opt-in "enable_dfs" option to the Mooncake JSON config. When set,
the connector initializes Mooncake's DFS backend during Store setup and
writes one DFS replica per object plus replica_num memory replicas
(kv_connector_extra_config, default 1). replica_num=0 stores objects
only in DFS and requires kvcache-ai/Mooncake#4637.

DFS is limited to the default tenant and is rejected for the
cross-encoder cache. DFS loads are split so each GET fits in
local_buffer_size, or in the SSD owner's staging buffer if smaller when
enable_offload is also set, since Mooncake reads SSD replicas first.

Tested with the Mooncake store unit tests and a 3FS run on Mooncake
main (d855a6ae) with replica_num 0 and 1.

Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: Daegyu Han <daegyu94.han@samsung.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ykwd ykwd ykwd approved these changes
@fcczzz fcczzz fcczzz approved these changes
@stmatengss stmatengss Awaiting requested review from stmatengss stmatengss is a code owner
@XucSh XucSh Awaiting requested review from XucSh XucSh is a code owner
@YiXR YiXR Awaiting requested review from YiXR YiXR is a code owner
@ShangmingCai ShangmingCai Awaiting requested review from ShangmingCai ShangmingCai is a code owner
@UNIDY2002 UNIDY2002 Awaiting requested review from UNIDY2002 UNIDY2002 is a code owner
@alogfans alogfans Awaiting requested review from alogfans alogfans is a code owner
@staryxchen staryxchen Awaiting requested review from staryxchen staryxchen is a code owner

Assignees

No one assigned

Labels

documentation Improvements or additions to documentation run-ci Store

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@2xdevv @ykwd @fcczzz