Skip to content

[Rust Frontend] Export Mooncake connector transfer metrics like Python - #63317

Merged
BugenZhao merged 2 commits into
vllm-project:mainfrom
BugenZhao:bz/rust-parity-mooncake-transfer-metrics
Oct 23, 2026
Merged

BugenZhao merged 2 commits into
vllm-project:mainfrom
BugenZhao:bz/rust-parity-mooncake-transfer-metrics

Conversation

@BugenZhao

Copy link
Copy Markdown
Member

Overview

#56939 added Prometheus metrics for the P2P MooncakeConnector. Under the Rust frontend these stats are dropped: the payload matches no typed KvConnectorStats variant, decodes as an unknown-child Multi, and records nothing. This PR decodes the payload and exports the six vllm:mooncake_* families with Python's names, labels, and buckets.

Claims

  • With a MooncakeConnector (bare or as a MultiConnector child), /metrics reports vllm:mooncake_xfer_time_seconds, vllm:mooncake_bytes_transferred, vllm:mooncake_num_descriptors (histograms) and vllm:mooncake_num_failed_transfers, vllm:mooncake_num_failed_recvs, vllm:mooncake_num_kv_expired_reqs (counters), labeled model_name and engine, as Python's MooncakePromMetrics does.
  • NIXL and Mooncake store metrics are unchanged.

Validation

Commands from .buildkite/scripts/run-rust-frontend-cargo-ci.sh (style-clippy and test), run from the repo root:

cargo fmt --manifest-path rust/Cargo.toml --all -- --check   # ok
cargo sort --workspace --check rust                          # ok
cargo deny --manifest-path rust/Cargo.toml --config rust/deny.toml check bans   # ok
cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets --all-features --locked -- -D warnings   # ok
cargo nextest run --manifest-path rust/Cargo.toml --workspace --all-features --locked --no-fail-fast
# 2295 tests run: 2295 passed, 13 skipped
  • metrics::tests::mooncake_transfer_stats_are_recorded_into_mooncake_metrics: a Python-shaped MooncakeConnector payload goes through decode and recording; the exposition is snapshotted.
  • metrics::tests::kv_connector_stats_are_recorded_into_mooncake_and_nixl_metrics (extended): a MultiConnector payload with a MooncakeConnector child records into the same families.
  • tests::client::python_msgpack_fixtures_match_rust_encoding: the Python fixture script now msgpack-encodes a MooncakeConnector payload, bare and as a MultiConnector child, and Rust decodes both into the new types.
  • With the variant placed after Multi (the old decode behavior), the bare-payload and fixture tests fail; without the MultiConnector recording branch, the extended test fails.

Details

  • Decode (engine-core-client protocol/stats.rs): MooncakeTransferStats mirrors MooncakeKVConnectorStats.data with deny_unknown_fields. It is its own KvConnectorStats variant, placed before Multi (which accepts any map; the enum now documents this order contract), and the MooncakeConnector child of MultiConnectorStats.
  • Register (vllm-metrics SchedulerMetrics): the six families, reusing the NIXL bucket constructors, which match Python's buckets.
  • Record (engine-core-client metrics.rs): per engine, next to record_nixl_stats.

This is the NIXL path with Mooncake's field set (no post_duration; num_failed_recvs instead of num_failed_notifications), with no new abstraction.

Difference kept: like the existing NIXL families in Rust, the Mooncake series are created for every engine at startup and report 0 when no Mooncake connector is configured; Python registers connector families only for the configured connector.

Not a duplicate: no open PR adds these families to Rust. #60856 (inactive since 2026-10-09) decodes NIXL connector stats in the same files and does not touch Mooncake.

AI assistance (Claude Code) was used to prepare this change; the human submitter reviewed every changed line.

🤖 Generated with Claude Code

https://claude.ai/code/session_012jJ3QaARmRe3jZJYTfHsCB


Pull Request Checklist
  • I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

  • AI assistance was used during the creation of this PR.

  • Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

  • Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

  • Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

  • Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

vllm-project#56939 added `MooncakePromMetrics` for the P2P
`MooncakeConnector`. Rust had no variant for its stats payload, so
it decoded as a MultiConnector map of unknown children and was dropped.
Decode it as `MooncakeTransferStats`, also as a MultiConnector child,
and export the six `vllm:mooncake_*` transfer families per engine,
reusing the NIXL bucket edges Python also uses.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
@BugenZhao
BugenZhao requested a review from njhill as a code owner October 23, 2026 09:17

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@BugenZhao

Copy link
Copy Markdown
Member Author

/ci run

@github-actions

Copy link
Copy Markdown

✅ Triggered Buildkite CI #96858 for commit 4c4abd4d0e27.

ℹ️ This PR is 1 commit behind upstream main. Merged upstream main into this branch as 4c4abd4d0e27 before starting CI. Pull the branch before pushing more changes.

@BugenZhao
BugenZhao merged commit 933f796 into vllm-project:main Oct 23, 2026
30 checks passed
bantianyue pushed a commit to bantianyue/vllm that referenced this pull request Oct 28, 2026
vllm-project#63317)

Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: vllm-agent <claw@inferact.ai>
@wangxiyuan wangxiyuan mentioned this pull request Oct 31, 2026
9 of 48 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

mooncake Mooncake KV-transfer / EC-transfer rust

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants