test(ci): correct the mm_router VRAM profile from 18.7 to the measured 7.6 GiB (OPS-8117)

[dmitry-tokarev-nv]

Summary

Eight e2e tests across tests/mm_router/test_router_rust_mm_router_e2e.py and test_router_rust_mm_frontend_decode_e2e.py reserve 18.7 GiB and use 7.4. Because 18.7 exceeds the multi-process budget (budget_multi = card x 0.85 = 19.12 GiB on the 22.49 GiB runner), nothing can ever co-reside with them, so all eight run strictly one at a time — 718s of forced serialization.

Gain: ~338s (5.6 min) off the vLLM GPU-parallel stage, from a two-line change.

The mark is provably wrong

All three files under tests/mm_router/ run the same model (Qwen/Qwen3-VL-2B-Instruct) with the same KV cap (requested_vllm_kv_cache_bytes(1_719_075_000)). That cap is not advisory — the orchestrator turns it into --kv-cache-memory-bytes N --gpu-memory-utilization 0.01, which pins the footprint and makes it card-independent. Same model + same cap = same footprint. Yet:

file	profiled_vram_gib
test_vllm_mm_router_e2e.py	7.6 (correct, unchanged)
test_router_rust_mm_router_e2e.py	18.7 → 7.6
test_router_rust_mm_frontend_decode_e2e.py	18.7 → 7.6

profiled_vram_gib is defined as the raw solo whole-GPU NVML peak with no safety factor, so the correct value is just the measured peak.

Validation — measured three ways

1. CI telemetry (run 31183861251, job 92889401992). All eight 18.7-marked mm_router tests were observed at exactly 7.6 GiB while running solo, 8–10 independent samples each.

Control group in the same stage, validating the method: test_vllm.py::test_reasoning_effort and ::test_tool_calling, also marked 18.7, measured 18.1 GiB. So the measurement is not biased low — those two marks are accurate and these eight were not.

2. Real runs on the gpu dev box (RTX 6000 Ada, idle 48 GiB card, same KV override CI applies), one test from each changed file:

test_router_rust_mm_logs_initialization      1 passed in 91.39s   PEAK 7598 MiB (48 samples)
test_frontend_decode_logs_decoded_bytes_...  1 passed in 87.15s   PEAK 7598 MiB (46 samples)

Identical to the byte on both — as the pinned KV cap predicts — and on a card more than twice CI's size, confirming the footprint does not scale with the card.

3. Sibling precedent: the correctly-marked third file has used 7.6 for the same workload all along.

Also checked on linux/arm64 (no GPU) that collection and marker selection are unchanged, and that the value the scheduler reads is now 7.6 for all eight:

7.6 GiB  kv_cap=1719075000  test_router_rust_mm_router_e2e.py::test_router_rust_mm_logs_initializat
7.6 GiB  kv_cap=1719075000  test_frontend_decode_e2e.py::test_frontend_decode_logs_decoded_by
...
8/11 tests collected (3 deselected)   # unchanged: the 3 deselected are the pre_merge sibling

Expected effect

Simulating _select_launches (the model reproduces the observed CI makespan exactly) against the vLLM GPU-parallel stage as it now stands, with the CPU-only unit tests already moved off it by #12826 — 61 e2e tests, 7,538s cumulative:

mm_router profile	stage makespan
18.7 (today)	4,240s
7.6 (measured)	3,902s

At 7.6 the eight tests pack two-wide (15.2 <= 19.12) instead of serializing; their durations total 718s today.

Risk

Low. 7.6 sits above the 7.42 measured peak, matches the long-standing sibling mark, and the scheduler applies a second independent gate against live nvidia-smi usage before admitting any test, so an under-estimate cannot silently overcommit the card.

Linear: https://linear.app/nvidia/issue/OPS-8117

🤖 Generated with Claude Code

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

[coderabbitai]

Walkthrough

Changes

MM router profiling configuration

Layer / File(s)	Summary
Update E2E profiling markers tests/mm_router/test_router_rust_mm_frontend_decode_e2e.py, tests/mm_router/test_router_rust_mm_router_e2e.py	The profiled GPU memory requirement changed from 18.7 GiB to 7.6 GiB. Comments document the measured peak, KV-cache configuration, and multi-process budget assumptions.

Estimated code review effort: 1 (Trivial) | ~5 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description gives detailed context, validation, impact, and risk, but it omits the required template sections and formal issue-link choice.	Add the required Overview, Details, reviewer-start, and Related Issues sections, and use the specified issue-link format or confirm that no issue is linked.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly identifies the corrected mm_router VRAM profile and the change from 18.7 to 7.6 GiB.

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

---

_{Comment @coderabbitai help to get the list of available commands.}

[dmitry-tokarev-nv]

Superseded by #12848, which carries the same two-line change rebuilt on current main (now that #12826 has merged) and re-validated end to end: one test from each changed file run on the gpu dev box, both peaking at 7,598 MiB, plus an arm64 collection check confirming the scheduler now reads 7.6 for all eight tests and that marker selection is unchanged.

[dmitry-tokarev-nv]

Correction to my earlier comment: this PR is the one to merge, not #12848 — it is already approved and #12848 has been closed. The two carried an identical change; I verified the diffs match exactly against current main, and this branch is MERGEABLE there (the change is confined to tests/mm_router/ and does not overlap #12826).

Description updated with the extra validation performed since: a second gpu-box run covering the other changed file, plus an arm64 collection check. No commits were pushed, so the approvals stand.

[dmitry-tokarev-nv]

Merged main into this branch (3ae6e875ce) so CI measures the real effect.

The branch was cut before #12826 landed, so its CI would have run the vLLM GPU-parallel stage with the 322 CPU-only unit tests still in it — inflating the baseline and burying this change's ~338s inside a much larger, unrelated swing. With main merged in, the stage under test is the one this change actually targets: ~61 e2e tests.

The merge is additive (no force-push) and the net diff against main is unchanged — still exactly the two marker lines:

$ git diff --stat origin/main...HEAD
 tests/mm_router/test_router_rust_mm_frontend_decode_e2e.py | 6 +++++-
 tests/mm_router/test_router_rust_mm_router_e2e.py          | 6 +++++-
 2 files changed, 10 insertions(+), 2 deletions(-)

Re-validated on the merged tree before pushing:

• gpu dev box (RTX 6000 Ada): test_router_rust_mm_logs_initialization passed in 84.0s, peak 7,596 MiB, with the profiled_vram_gib(7.6) mark confirmed present in the tree under test. Consistent with the earlier 7,598 MiB runs on both changed files.
• linux/arm64, no GPU: all eight mm_router tests report 7.6 GiB to the scheduler, and test(ci): reclassify CPU-only vllm unit tests from gpu_1 to gpu_0 (OPS-8113) #12826's lane is healthy on the merged tree (1063 passed, 6 skipped).

The A/B to read once CI finishes. Compare the vllm-runtime / Test cuda13.0, amd64 job's GPU tests (parallel) step:

• baseline, post-test(ci): reclassify CPU-only vllm unit tests from gpu_1 to gpu_0 (OPS-8113) #12826 without this change: post-merge run 31212953654 (a274300453)
• with this change: this PR's run on 3ae6e875ce

Those two differ only by these two marker lines, so the delta is the whole story. Expect ~4,240s -> ~3,902s.

Note this needs a fresh /ok to test 3ae6e875ce for the new head.