test(ci): correct the mm_router VRAM profile from 18.7 to the measured 7.6 GiB (OPS-8117)

[dmitry-tokarev-nv]

Summary

Eight e2e tests across tests/mm_router/test_router_rust_mm_router_e2e.py and test_router_rust_mm_frontend_decode_e2e.py reserve 18.7 GiB and use 7.4. Because 18.7 exceeds the multi-process budget (budget_multi = card x 0.85 = 19.12 GiB on the 22.49 GiB runner), nothing can ever co-reside with them, so all eight run strictly one at a time — 718s of forced serialization.

Gain: ~338s (5.6 min) off the vLLM GPU-parallel stage, from a two-line change. Branched off current main, so this measures against the stage as it stands now that #12826 has moved the CPU-only unit tests off it.

The mark is provably wrong

All three files under tests/mm_router/ run the same model (Qwen/Qwen3-VL-2B-Instruct) with the same KV cap (requested_vllm_kv_cache_bytes(1_719_075_000)). That cap is not advisory — the orchestrator turns it into --kv-cache-memory-bytes N --gpu-memory-utilization 0.01, which pins the footprint and makes it card-independent. Same model + same cap = same footprint. Yet:

file	profiled_vram_gib
test_vllm_mm_router_e2e.py	7.6 (correct, unchanged)
test_router_rust_mm_router_e2e.py	18.7 → 7.6
test_router_rust_mm_frontend_decode_e2e.py	18.7 → 7.6

profiled_vram_gib is defined as the raw solo whole-GPU NVML peak with no safety factor, so the correct value is just the measured peak.

Validation — measured three ways

1. CI telemetry (run 31183861251, job 92889401992). All eight 18.7-marked mm_router tests were observed at exactly 7.6 GiB while running solo, 8–10 independent samples each.

Control group in the same stage, validating the method: test_vllm.py::test_reasoning_effort and ::test_tool_calling, also marked 18.7, measured 18.1 GiB. So the measurement is not biased low — those two marks are accurate and these eight were not.

2. Real runs on the gpu dev box (RTX 6000 Ada, idle 48 GiB card, same KV override CI applies), one test from each changed file:

test_router_rust_mm_logs_initialization      1 passed in 91.39s   PEAK 7598 MiB (48 samples)
test_frontend_decode_logs_decoded_bytes_...  1 passed in 87.15s   PEAK 7598 MiB (46 samples)

Identical to the byte on both — as the pinned KV cap predicts — and on a card more than twice CI's size, confirming the footprint does not scale with the card.

3. Sibling precedent: the correctly-marked third file has used 7.6 for the same workload all along.

Also checked on linux/arm64 (no GPU) that collection and marker selection are unchanged, and that the value the scheduler reads is now 7.6 for all eight:

7.6 GiB  kv_cap=1719075000  test_router_rust_mm_router_e2e.py::test_router_rust_mm_logs_initializat
7.6 GiB  kv_cap=1719075000  test_frontend_decode_e2e.py::test_frontend_decode_logs_decoded_by
...
8/11 tests collected (3 deselected)   # unchanged: the 3 deselected are the pre_merge sibling

Expected effect

Simulating _select_launches (the model reproduces the observed CI makespan exactly) against the current vLLM GPU-parallel stage — 61 e2e tests, 7,538s cumulative:

mm_router profile	stage makespan
18.7 (today)	4,240s
7.6 (measured)	3,902s

At 7.6 the eight tests pack two-wide (15.2 <= 19.12) instead of serializing; their durations total 718s today.

Risk

Low. 7.6 sits above the 7.42 measured peak, matches the long-standing sibling mark, and the scheduler applies a second independent gate against live nvidia-smi usage before admitting any test, so an under-estimate cannot silently overcommit the card.

Linear: https://linear.app/nvidia/issue/OPS-8117

🤖 Generated with Claude Code

---

Summary by CodeRabbit

• Tests
  • Updated multimodal router end-to-end test resource markers to reflect the measured peak VRAM usage of 7.6 GiB.

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 212b554b-ebda-413f-a684-fdb3981cc649

📥 Commits

Reviewing files that changed from the base of the PR and between a274300 and ba5f073.

📒 Files selected for processing (2)

• tests/mm_router/test_router_rust_mm_frontend_decode_e2e.py
• tests/mm_router/test_router_rust_mm_router_e2e.py

---

Walkthrough

The pull request updates two Rust multimodal router E2E tests. Their profiled VRAM markers change from 18.7 to 7.6 GiB. Comments document the measured peak and test configuration.

Changes

Multimodal router VRAM calibration

Layer / File(s)	Summary
Update E2E VRAM markers tests/mm_router/test_router_rust_mm_frontend_decode_e2e.py, tests/mm_router/test_router_rust_mm_router_e2e.py	Both test suites use a 7.6 GiB profiled_vram_gib marker and document the measured solo peak, KV-cap configuration, model parity, and prior multi-process budget issue.

Estimated code review effort: 1 (Trivial) | ~2 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the CI VRAM profile correction, the affected value, and the related issue.
Description check	✅ Passed	The description is detailed and on-topic, with scope, validation, expected impact, risk, affected files, and a linked issue.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

---

_{Comment @coderabbitai help to get the list of available commands.}

[dmitry-tokarev-nv]

Closing in favour of #12842, which is already approved and carries an identical change. Verified identical against current main:

$ diff <(git diff origin/main...origin/dtokarev/ops-8115-mm-router-vram-profile -- tests/) \
       <(git diff origin/main...origin/dtokarev/ops-8117-mm-router-vram-profile-v2 -- tests/)
IDENTICAL

Both produce the same 2 files / 10 insertions and touch nothing else; #12842 is MERGEABLE against current main. The only difference was the merge base, which does not matter here since the change is confined to tests/mm_router/ and does not overlap #12826.

The extra validation done for this PR (a second gpu-box run covering test_router_rust_mm_frontend_decode_e2e.py, and the arm64 collection check) has been folded into #12842's description.