Repository navigation
Conversation
vllm-project#56939 added `MooncakePromMetrics` for the P2P `MooncakeConnector`. Rust had no variant for its stats payload, so it decoded as a MultiConnector map of unknown children and was dropped. Decode it as `MooncakeTransferStats`, also as a MultiConnector child, and export the six `vllm:mooncake_*` transfer families per engine, reusing the NIXL bucket edges Python also uses. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Isotr0py
approved these changes
Oct 23, 2026
Member
Author
|
/ci run |
|
✅ Triggered Buildkite CI #96858 for commit ℹ️ This PR is 1 commit behind upstream |
bantianyue
pushed a commit
to bantianyue/vllm
that referenced
this pull request
Oct 28, 2026
vllm-project#63317) Signed-off-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: vllm-agent <claw@inferact.ai>
9 of 48 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
#56939 added Prometheus metrics for the P2P
MooncakeConnector. Under the Rust frontend these stats are dropped: the payload matches no typedKvConnectorStatsvariant, decodes as an unknown-childMulti, and records nothing. This PR decodes the payload and exports the sixvllm:mooncake_*families with Python's names, labels, and buckets.Claims
MooncakeConnector(bare or as a MultiConnector child),/metricsreportsvllm:mooncake_xfer_time_seconds,vllm:mooncake_bytes_transferred,vllm:mooncake_num_descriptors(histograms) andvllm:mooncake_num_failed_transfers,vllm:mooncake_num_failed_recvs,vllm:mooncake_num_kv_expired_reqs(counters), labeledmodel_nameandengine, as Python'sMooncakePromMetricsdoes.Validation
Commands from
.buildkite/scripts/run-rust-frontend-cargo-ci.sh(style-clippyandtest), run from the repo root:metrics::tests::mooncake_transfer_stats_are_recorded_into_mooncake_metrics: a Python-shapedMooncakeConnectorpayload goes through decode and recording; the exposition is snapshotted.metrics::tests::kv_connector_stats_are_recorded_into_mooncake_and_nixl_metrics(extended): a MultiConnector payload with aMooncakeConnectorchild records into the same families.tests::client::python_msgpack_fixtures_match_rust_encoding: the Python fixture script now msgpack-encodes aMooncakeConnectorpayload, bare and as a MultiConnector child, and Rust decodes both into the new types.Multi(the old decode behavior), the bare-payload and fixture tests fail; without the MultiConnector recording branch, the extended test fails.Details
engine-core-clientprotocol/stats.rs):MooncakeTransferStatsmirrorsMooncakeKVConnectorStats.datawithdeny_unknown_fields. It is its ownKvConnectorStatsvariant, placed beforeMulti(which accepts any map; the enum now documents this order contract), and theMooncakeConnectorchild ofMultiConnectorStats.vllm-metricsSchedulerMetrics): the six families, reusing the NIXL bucket constructors, which match Python's buckets.engine-core-clientmetrics.rs): per engine, next torecord_nixl_stats.This is the NIXL path with Mooncake's field set (no
post_duration;num_failed_recvsinstead ofnum_failed_notifications), with no new abstraction.Difference kept: like the existing NIXL families in Rust, the Mooncake series are created for every engine at startup and report
0when no Mooncake connector is configured; Python registers connector families only for the configured connector.Not a duplicate: no open PR adds these families to Rust. #60856 (inactive since 2026-10-09) decodes NIXL connector stats in the same files and does not touch Mooncake.
AI assistance (Claude Code) was used to prepare this change; the human submitter reviewed every changed line.
🤖 Generated with Claude Code
https://claude.ai/code/session_012jJ3QaARmRe3jZJYTfHsCB
Pull Request Checklist
I used vLLM's
/pr-checklistskill. (Mandatory for agents, optional for humans).AI assistance was used during the creation of this PR.
Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.
Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.
Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.
Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.