Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
a603396
Add ECMooncakeConnector for encoder cache over Mooncake TransferEngine
stmatengss May 3, 2026
0237964
Add Mooncake transfer reliability knobs and EC connector sync
stmatengss May 25, 2026
b6d1445
Fix issues with latest main
gty111 Aug 12, 2026
f942d70
[EC] Manage Mooncake registration lifetime
gty111 Aug 12, 2026
320815b
Batch Mooncake embedding transfers
gty111 Aug 13, 2026
578126a
Clean up code
gty111 Aug 13, 2026
c58f20d
Add consumer_buffer_pool
gty111 Aug 14, 2026
938f874
Merge branch 'main' into add-mooncake-ec-connector
gty111 Aug 14, 2026
5567c37
Merge remote-tracking branch 'origin/main' into add-mooncake-ec-conne…
gty111 Aug 15, 2026
f50448c
Fix stale encoder cache eviction notifications
gty111 Aug 16, 2026
456b7dc
Support async load and save for PUSH mode
gty111 Aug 17, 2026
10db3ae
Support consumer TP and PP
gty111 Aug 18, 2026
b429844
Fix consumer TP>1
gty111 Aug 18, 2026
72135d5
Give each consumer replica its own block of control ports
gty111 Aug 19, 2026
af3e699
Support DP
gty111 Aug 19, 2026
7c0cd23
Merge remote-tracking branch 'origin/main' into add-mooncake-ec-conne…
gty111 Aug 19, 2026
e3d84ea
Merge branch 'main' into add-mooncake-ec-connector
gty111 Aug 20, 2026
35a88b0
Merge branch 'main' into add-mooncake-ec-connector
gty111 Aug 21, 2026
e41cf6a
Merge branch 'main' into add-mooncake-ec-connector
gty111 Aug 22, 2026
d4ff1b4
[Bugfix][EPD] Fix DeepStack encoder cache reservation sizing (#4)
jiaran-king Aug 24, 2026
bd9a3e5
Merge remote-tracking branch 'origin/main' into add-mooncake-ec-conne…
gty111 Aug 24, 2026
720f98f
[Bugfix] Prevent duplicate Mooncake push metadata across scheduler st…
jiaran-king Aug 27, 2026
f67f390
[Bugfix] Ignore late Mooncake ready events for cancelled transfers (#6)
jiaran-king Aug 28, 2026
4151de2
[Bugfix] Sweep Mooncake cancel records in deadline order
gty111 Aug 28, 2026
121f2f6
Merge branch 'main' into add-mooncake-ec-connector
gty111 Aug 28, 2026
e5aa2bc
Fix Mooncake EPD proxy flag in integration script (#7)
jiangkuaixue123 Aug 29, 2026
2f44446
[EPD] Make the proxy's per-request logging opt-in
gty111 Aug 29, 2026
8ab409c
Merge origin/main into add-mooncake-ec-connector
gty111 Sep 1, 2026
0b319fd
[EPD] Refactor ECMooncakeConnector internals (#9)
jiangkuaixue123 Sep 1, 2026
a8cb381
Merge branch 'main' into add-mooncake-ec-connector
gty111 Sep 2, 2026
7d03432
[EPD] Fix the EC transfer-id desync that leaks consumer reservations
gty111 Sep 3, 2026
682955d
[EPD] Simplify Mooncake EC connector internals (#10)
jiaran-king Sep 4, 2026
1faf3ee
[EPD] Keep local_cache_hashes on the public EC connector hook
gty111 Sep 4, 2026
fd60550
Merge branch 'main' into add-mooncake-ec-connector
gty111 Sep 4, 2026
f073b1d
[EPD] Mirror the local encoder cache in the Mooncake scheduler
gty111 Sep 4, 2026
8808bfb
[EPD] Address Mooncake EC connector review findings
gty111 Sep 4, 2026
bee2142
[EPD] Harden the Mooncake EC connector against control-plane faults
gty111 Sep 4, 2026
c1c9545
[EPD] Gate both WAITING paths on encoder-cache availability
gty111 Sep 4, 2026
875fe57
[EPD] Harden Mooncake EC transfers and batch asynchronous pushes
gty111 Sep 5, 2026
50eb506
[EPD] Replace Mooncake EC unit tests with TCP E2E CI
gty111 Sep 6, 2026
a88a755
[EPD] Align Mooncake EC test control addresses
gty111 Sep 7, 2026
1010f8e
Merge branch 'main' into add-mooncake-ec-connector
gty111 Sep 7, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions .buildkite/test_areas/disaggregated_mooncake.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,57 @@ steps:
commands:
- bash /vllm-workspace/.buildkite/scripts/install-kv-connectors.sh
- bash v1/kv_connector/mooncake_integration/config_sweep_accuracy_test.sh

- label: ":nvidia: (L4) Mooncake EC TCP E2E"
key: mooncake-ec-tcp-e2e-2-gpus
timeout_in_minutes: 30
working_dir: "/vllm-workspace"
device: l4
num_devices: 2
source_file_dependencies:
- vllm/distributed/ec_transfer/
- vllm/config/ec_transfer.py
- vllm/config/multimodal.py
- vllm/config/vllm.py
- vllm/multimodal/
- vllm/v1/core/encoder_cache_manager.py
- vllm/v1/core/sched/
- vllm/v1/engine/
- vllm/v1/worker/ec_connector_model_runner_mixin.py
- vllm/v1/worker/gpu_model_runner.py
- vllm/v1/worker/gpu_worker.py
- vllm/v1/worker/gpu/ec_connector.py
- vllm/v1/worker/gpu/model_runner.py
- vllm/v1/worker/gpu/mm/
- examples/disaggregated/disaggregated_encoder/
- tests/v1/ec_connector/
- requirements/kv_connectors.txt
env:
PYTHON_BIN: "/vllm-workspace/.venv/bin/python"
PYTHONUNBUFFERED: "1"
VLLM_HOST_IP: "127.0.0.1"
VLLM_USE_V2_MODEL_RUNNER: "1"
MOONCAKE_EC_PROTOCOL: "tcp"
USE_MM_PROMPTS: "1"
SKIP_BASELINE: "0"
CONCURRENCY: "3"
REPEAT: "2"
LOG_PATH: "/tmp/mooncake-ec-e2e"
BASELINE_FILE: "/tmp/mooncake-ec-e2e/baseline.json"
commands:
- uv venv --system-site-packages --python 3.12 .venv
- |
uv pip install --python .venv/bin/python "$(.venv/bin/python - <<'PY'
from pathlib import Path
import torch

requirement = next(
line for line in Path("requirements/kv_connectors.txt").read_text().splitlines()
if line.startswith("mooncake-transfer-engine ")
)
if torch.version.cuda.split(".")[0] == "13":
requirement = requirement.replace("mooncake-transfer-engine", "mooncake-transfer-engine-cuda13")
print(requirement)
PY
)"
- bash tests/v1/ec_connector/integration/run_epd_mooncake_ec_full_pipeline.sh
Loading
Loading