Skip to content

[MRV2] Worker-Level Timings - #50919

Draft
benchislett wants to merge 1 commit into
vllm-project:mainfrom
benchislett:mrv2-device-metrics
Draft

benchislett wants to merge 1 commit into
vllm-project:mainfrom
benchislett:mrv2-device-metrics

Conversation

@benchislett

Copy link
Copy Markdown
Member

Purpose

  1. With minimal overhead, profile the model runner and speculative decoding proposer (if present) timings
  2. Expose the timings via metrics endpoint and also significantly improve the UX and reliability of the iteration-level logging details (see log_worker_iteration_details)
  3. Establish reliable statistics for introspective profiling-based tuning such as adaptive verification

Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
@mergify

mergify Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Documentation preview: https://vllm--50919.org.readthedocs.build/en/50919/

@mergify mergify Bot added documentation Improvements or additions to documentation mrv2 Model Runner V2 specific labels Aug 3, 2026
@mergify

mergify Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @benchislett.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation mrv2 Model Runner V2 specific needs-rebase scheduler

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant