[Misc] Add P2P transfer stats gauges to Promethus

[chickeyton]

An answer to TTFT Routing of vllm for a more accurate TTFT estimation

It measure the average p2p transfer time of chunks (from CPU and Disk) and report by the Promethus mechanism

Actual Code Changes

1. Add update functions to LMCStatsMonitor for recording P2P transfer time from CPU/Disk for each cache chunk
2. Invoke update functions of LMCStatsMonitor in NaiveDistributedServer after transferring a cache chunk to other peer.
3. Add lmcache:avg_p2p_transfer_time_from_disk and lmcache:avg_p2p_transfer_time_from_cpu gauges to Promethus for reporting

Testing

1. start vllm with LMCache connector:

mkdir /tmp/vllm_prometheus

PROMETHEUS_MULTIPROC_DIR=/tmp/vllm_prometheus \
vllm serve Qwen/Qwen3-0.6B \
--gpu-memory-utilization 0.8 \
--kv-transfer-config '{"kv_connector":"LMCacheConnectorV1"}'

2. After startup success, query /metrics:

curl http://localhost:8080/metrics | grep lmcache:avg_p2p_transfer_time_from

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 49027  100 49027    0     0  1680k      0 --:--:-- --:--:-- --:--:-- 1709k
# HELP lmcache:avg_p2p_transfer_time_from_disk Average p2p transfer time from disk (seconds) of a chunk
# TYPE lmcache:avg_p2p_transfer_time_from_disk gauge
lmcache:avg_p2p_transfer_time_from_disk{model_name="Qwen/Qwen3-0.6B",worker_id="0"} 0.0
# HELP lmcache:avg_p2p_transfer_time_from_cpu Average p2p transfer time from CPU (seconds) of a chunk
# TYPE lmcache:avg_p2p_transfer_time_from_cpu gauge
lmcache:avg_p2p_transfer_time_from_cpu{model_name="Qwen/Qwen3-0.6B",worker_id="0"} 0.0

---

PR Checklist (Click to Expand)

Thank you for your contribution to LMCache! Before submitting the pull request, please ensure the PR meets the following criteria. This helps us maintain the code quality and improve the efficiency of the review process.

PR Title and Classification

Please try to classify PRs for easy understanding of the type of changes. The PR title is prefixed appropriately to indicate the type of change. Please use one of the following:

• [Bugfix] for bug fixes.
• [CI/Build] for build or continuous integration improvements.
• [Doc] for documentation fixes and improvements.
• [Model] for adding a new model or improving an existing model. Model name should appear in the title.
• [Core] for changes in the core LMCache logic (e.g., LMCacheEngine, Backend etc.)
• [Misc] for PRs that do not fit the above categories. Please use this sparingly.

Note: If the PR spans more than one category, please include all relevant prefixes.

Code Quality

The PR need to meet the following code quality standards:

• The code need to be well-documented to ensure future contributors can easily understand the code.
• Please include sufficient unit tests to ensure the change is stay correct and robust. The unit and integration tests will always run and our comprehensive test will be triggered after the "full" label is tagged onto a PR.

What to Expect for the Reviews

We aim to address all PRs in a timely manner. If no one reviews your PR within 5 days, please @-mention one of KuntaiDu, ApostaC or YaoJiayi.

[gemini-code-assist]

Summary of Changes

Hello @chickeyton, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the observability of the LMCache system by adding detailed performance metrics for peer-to-peer cache chunk transfers. It provides insights into the average time taken for chunks to be transferred from both disk and CPU sources, making these statistics available through Prometheus for better monitoring and performance analysis of the distributed caching mechanism.

Highlights

• New P2P Transfer Time Metrics: Introduced new metrics to track the average peer-to-peer transfer time for cache chunks, differentiating between transfers originating from disk and CPU.
• LMCStatsMonitor Integration: Added update functions to LMCStatsMonitor to record these new p2p transfer times and integrated their calculation into the statistics aggregation process.
• Prometheus Reporting: Exposed the new average p2p transfer times as Prometheus gauges, allowing for external monitoring and analysis of these critical performance indicators.
• Distributed Server Instrumentation: Implemented the logic within NaiveDistributedServer to measure and report p2p transfer times whenever a cache chunk is retrieved, categorizing them by their source (disk or CPU).

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request adds metrics for P2P transfer times to Prometheus, which is a useful addition for observability. The changes are mostly correct, but I've found a bug in the stats calculation and a performance issue due to redundant lookups. My review includes a fix for the bug and a suggestion to improve performance. Overall, good work on enhancing the monitoring capabilities.

[YaoJiayi]

@chickeyton This is certainly useful. But we might want to hold this for several days as I'm refactoring the P2P code

[chickeyton]

@chickeyton This is certainly useful. But we might want to hold this for several days as I'm refactoring the P2P code

I see, I also found the p2p transfer seems not working after rebase

[chickeyton]

No more needed by TTFT Routing