[Bugfix][MRV2] Support encoder timing stats in model runner V2 by guan404ming · Pull Request #50020 · vllm-project/vllm

guan404ming · 2026-07-27T17:19:50Z

Purpose

vllm bench mm-processor crashes on model runner V2: get_encoder_timing_stats only exists in V1
Port it to V2: timing lives on EncoderRunner, gated on enable_mm_processor_stats; EncoderTimingStats moves to worker/utils.py for sharing
Not a duplicate: no open PR touches MRV2 encoder timing

Test Plan

pytest tests/v1/worker/test_encoder_runner.py — adds registry accumulate/clear test
vllm bench mm-processor --model Qwen/Qwen2.5-VL-3B-Instruct --dataset-name random-mm --num-prompts 10 on Modal L4 (MRV2 default)

Test Result

8 passed
Before: AttributeError: 'GPUModelRunner' object has no attribute 'get_encoder_timing_stats'
After: benchmark completes with encoder stats populated

Stage	Mean	Median	P99
encoder_forward_ms	138.99	183.15	201.11
num_encoder_calls	1.00	1.00	1.00

AI assistance was used for this change (Claude); every line was reviewed by the submitter.

claude

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

guan404ming · 2026-07-27T17:21:45Z

cc @Isotr0py

mergify · 2026-08-05T05:22:51Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @guan404ming.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Isotr0py · 2026-08-05T05:46:06Z

-        self, mm_kwargs: list[tuple[str, MultiModalKwargsItem]]
+        self,
+        mm_kwargs: list[tuple[str, MultiModalKwargsItem]],
+        request_ids: Collection[str] | None = None,
    ) -> list[torch.Tensor]:


I would like to keep execute_mm_encoder only accept mm_kwargs to make sure the function is clean enough. Perhaps you can implement a context manager for timing instead of patching the function:

vllm/vllm/v1/worker/gpu_model_runner.py

Lines 7807 to 7849 in 7794b1e

@contextmanager

def timed_encoder_operation(

self,

should_time: bool,

group_lora_refs: list[tuple[str, Any]],

current_item_idx: int,

num_items: int,

):

"""

Context manager to time encoder forward operations.

Args:

should_time: Whether timing is enabled

group_lora_refs: Full list of (request_id, pos_info) tuples

current_item_idx: Starting index for this group

num_items: Number of items in this group

"""

if not should_time:

yield

return

group_refs = group_lora_refs[current_item_idx : current_item_idx + num_items]

group_request_ids = {req_id for req_id, _ in group_refs}

torch.accelerator.synchronize()

start_time = time.perf_counter()

try:

yield

finally:

torch.accelerator.synchronize()

elapsed = time.perf_counter() - start_time

per_request_time = elapsed / max(len(group_request_ids), 1)

with self._encoder_timing_lock:

for req_id in group_request_ids:

if req_id not in self.encoder_timing_registry:

self.encoder_timing_registry[req_id] = EncoderTimingStats()

stats = self.encoder_timing_registry[req_id]

stats.encoder_forward_secs += per_request_time

stats.num_encoder_calls += 1

Good point. Reverted execute_mm_encoder to only accept mm_kwargs, and moved the timing into a timed_encoder_operation context manager on EncoderRunner, mirroring the V1 pattern you linked. Callers now wrap the encoder call with it. PTAL, thanks!

guan404ming · 2026-08-06T09:46:08Z

Hi @Isotr0py could you help trigger the buildkite ci? It seems to be inactive for a while. Thanks!

Isotr0py · 2026-08-06T09:52:51Z

/ci run

github-actions · 2026-08-06T09:53:09Z

✅ Triggered Buildkite CI #82663 for commit 186691282a0d.

guan404ming · 2026-08-06T09:54:03Z

Thanks!

guan404ming · 2026-08-07T03:31:38Z

/ci run

github-actions · 2026-08-07T03:31:54Z

✅ Triggered Buildkite CI #82774 for commit dbf9f4712526.

mergify · 2026-08-07T03:42:18Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @guan404ming.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>

guan404ming · 2026-08-07T04:32:14Z

/ci run

github-actions · 2026-08-07T04:32:30Z

✅ Triggered Buildkite CI #82781 for commit e9b51dcbf4f5.

guan404ming · 2026-08-07T06:24:04Z

Hi @Isotr0py I just pushed a commit to resolve the conflict. Could you help enable merge again? Thanks!

guan404ming · 2026-08-08T06:06:07Z

/ci run

github-actions · 2026-08-08T06:06:23Z

✅ Triggered Buildkite CI #82983 for commit 5be9679d7d98.

guan404ming · 2026-08-11T01:41:48Z

/ci run

github-actions · 2026-08-11T01:42:03Z

✅ Triggered Buildkite CI #83261 for commit 9ef0ae4f6d71.

guan404ming · 2026-08-11T14:25:05Z

/ci run

guan404ming · 2026-08-11T14:25:36Z

Hi @Isotr0py just a gentle ping. I pushed a commit to resolve the conflict. Could you help enable merge again? Thanks!

github-actions · 2026-08-11T14:26:01Z

✅ Triggered Buildkite CI #83364 for commit ec4fa70d5c9b.

guan404ming · 2026-08-11T16:20:30Z

Thanks!

…project#50020) Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

mergify Bot added v1 bug Something isn't working labels Jul 27, 2026

guan404ming marked this pull request as ready for review July 27, 2026 17:21

guan404ming requested review from WoosukKwon, njhill and yewentao256 as code owners July 27, 2026 17:21

claude Bot reviewed Jul 27, 2026

View reviewed changes

Isotr0py self-assigned this Jul 28, 2026

mergify Bot added the mrv2 Model Runner V2 specific label Jul 31, 2026

guan404ming force-pushed the fix/mrv2-encoder-timing-stats branch from 312a368 to f452563 Compare August 4, 2026 16:24

mergify Bot added the needs-rebase label Aug 5, 2026

Isotr0py reviewed Aug 5, 2026

View reviewed changes

guan404ming force-pushed the fix/mrv2-encoder-timing-stats branch from f452563 to 4cd9ef7 Compare August 5, 2026 09:19

mergify Bot removed the needs-rebase label Aug 5, 2026

Isotr0py approved these changes Aug 5, 2026

View reviewed changes

Isotr0py enabled auto-merge (squash) August 5, 2026 13:15

github-actions Bot added the ready ONLY add when PR is ready to merge/full CI is needed label Aug 5, 2026

mergify Bot added the needs-rebase label Aug 7, 2026

guan404ming added 3 commits August 7, 2026 11:58

Add encoder timing stats to model runner V2

2c8561e

Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>

Fix trailing blank line

69b4f9d

Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>

Use context manager for encoder timing

e9b51dc

Signed-off-by: Guan-Ming (Wesley) Chiu <105915352+guan404ming@users.noreply.github.com>

auto-merge was automatically disabled August 7, 2026 04:00
Head branch was pushed to by a user without write access

guan404ming force-pushed the fix/mrv2-encoder-timing-stats branch from dbf9f47 to e9b51dc Compare August 7, 2026 04:00

mergify Bot removed the needs-rebase label Aug 7, 2026

Merge branch 'main' into fix/mrv2-encoder-timing-stats

5be9679

Merge branch 'main' into fix/mrv2-encoder-timing-stats

9ef0ae4

Merge branch 'main' into fix/mrv2-encoder-timing-stats

ec4fa70

Isotr0py merged commit 457a5f3 into vllm-project:main Aug 11, 2026
94 checks passed

guan404ming deleted the fix/mrv2-encoder-timing-stats branch August 11, 2026 16:20

	@contextmanager
	def timed_encoder_operation(
	self,
	should_time: bool,
	group_lora_refs: list[tuple[str, Any]],
	current_item_idx: int,
	num_items: int,
	):
	"""
	Context manager to time encoder forward operations.

	Args:
	should_time: Whether timing is enabled
	group_lora_refs: Full list of (request_id, pos_info) tuples
	current_item_idx: Starting index for this group
	num_items: Number of items in this group
	"""
	if not should_time:
	yield
	return

	group_refs = group_lora_refs[current_item_idx : current_item_idx + num_items]
	group_request_ids = {req_id for req_id, _ in group_refs}

	torch.accelerator.synchronize()
	start_time = time.perf_counter()

	try:
	yield
	finally:
	torch.accelerator.synchronize()
	elapsed = time.perf_counter() - start_time

	per_request_time = elapsed / max(len(group_request_ids), 1)

	with self._encoder_timing_lock:
	for req_id in group_request_ids:
	if req_id not in self.encoder_timing_registry:
	self.encoder_timing_registry[req_id] = EncoderTimingStats()

	stats = self.encoder_timing_registry[req_id]
	stats.encoder_forward_secs += per_request_time
	stats.num_encoder_calls += 1

Uh oh!

Conversation

guan404ming commented Jul 27, 2026

Purpose

Test Plan

Test Result

Uh oh!

claude Bot left a comment

Choose a reason for hiding this comment

Claude Code Review

Uh oh!

guan404ming commented Jul 27, 2026

Uh oh!

mergify Bot commented Aug 5, 2026

Uh oh!

Isotr0py Aug 5, 2026

Choose a reason for hiding this comment

Uh oh!

guan404ming Aug 5, 2026

Choose a reason for hiding this comment

Uh oh!

guan404ming commented Aug 6, 2026

Uh oh!

Isotr0py commented Aug 6, 2026

Uh oh!

github-actions Bot commented Aug 6, 2026

Uh oh!

guan404ming commented Aug 6, 2026

Uh oh!

guan404ming commented Aug 7, 2026

Uh oh!

github-actions Bot commented Aug 7, 2026

Uh oh!

mergify Bot commented Aug 7, 2026

Uh oh!

guan404ming commented Aug 7, 2026

Uh oh!

github-actions Bot commented Aug 7, 2026

Uh oh!

guan404ming commented Aug 7, 2026

Uh oh!

guan404ming commented Aug 8, 2026

Uh oh!

github-actions Bot commented Aug 8, 2026

Uh oh!

guan404ming commented Aug 11, 2026

Uh oh!

github-actions Bot commented Aug 11, 2026

Uh oh!

guan404ming commented Aug 11, 2026

Uh oh!

guan404ming commented Aug 11, 2026

Uh oh!

github-actions Bot commented Aug 11, 2026

Uh oh!

Uh oh!

guan404ming commented Aug 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants