fix: restore VLM nightly regression coverage

[mickqian]

Summary

• keep single-node CUDA IPC multimodal feature transport opt-in
• preserve automatic CUDA VMM selection for validated multi-node MNNVL deployments
• refresh VLM MMMU accuracy and latency baselines for the 1024-token CoT evaluation budget

Root cause

PR #27327 increased the MMMU CoT generation budget from 30 to 1024 tokens, but the latency thresholds still described the old 30-token workload. Nightly results moved to a new stable latency plateau immediately after that change, so the old thresholds no longer measured engine regressions.

Pixtral's score has ranged from 0.33 to 0.37 on the fixed 100-sample suite since that change. A 0.36 floor therefore flakes on normal one- or two-question variation; this PR uses 0.32 as the regression floor.

GLM-4.5V-FP8 also had a separate real regression after single-node CUDA IPC became the default. Its fixed 1 GiB feature pool reduced the GLM nightly KV cache from 25,998 to 14,088 tokens, reducing output throughput from about 687 to 549 tokens/s and increasing latency from about 107 to 131 seconds. The latest run reached 148 seconds. Restoring CPU as the single-node default recovers the KV-cache headroom while keeping CUDA IPC available through --mm-feature-transport=cuda_ipc.

The GLM latency threshold is calibrated to the pre-regression 100-119 second plateau with headroom at 140 seconds, so a 208-second result still fails.

Validation

• all changed-file pre-commit hooks pass
• Python byte compilation passes
• git diff --check passes
• targeted unit-test collection is blocked on the local macOS environment because its Transformers build lacks transformers.image_processing_backends; Linux/H100 CI is required for runtime validation

---

CI States

Latest PR Test (Base): ❌ Run #31668017794
Latest PR Test (Extra): ❌ Run #31668017661

[mickqian]

/tag-run-ci-label

[mickqian]

/tag-and-rerun-ci

[mickqian]

/rerun-test

[github-actions]

⛔ Please specify a test: /rerun-test <file>::<TestClass.test_method>

Examples:

• /rerun-test test/registered/core/test_srt_endpoint.py::TestSRTEndpoint.test_simple_decode
• /rerun-test registered/core/test_srt_endpoint.py::TestSRTEndpoint
• /rerun-test test_srt_endpoint.py
• /rerun-test test_a.py test_b.py test_c.py (multiple tests)
• /rerun-test test_*backend*.py (wildcard — reruns every matching file; wrap the pattern in backticks so GitHub keeps the * literal)

[hnyls2002]

/rerun-test test/registered/eval/test_vlms_mmmu_eval.py

[github-actions]

Results for /rerun-test test/registered/eval/test_vlms_mmmu_eval.py:

🚀 2-gpu-h100 (1 test): ❌ View workflow run

cd test/ && python3 registered/eval/test_vlms_mmmu_eval.py

[github-actions]

🚀 Dispatching /rerun-test test/registered/eval/test_vlms_mmmu_eval.py...

[mickqian]

/rerun-test test/registered/eval/test_vlms_mmmu_eval.py

[github-actions]

Results for /rerun-test test/registered/eval/test_vlms_mmmu_eval.py:

🚀 2-gpu-h100 (1 test): ❌ View workflow run

cd test/ && python3 registered/eval/test_vlms_mmmu_eval.py

[mickqian]

/rerun-test test/registered/eval/test_vlms_mmmu_eval.py

[github-actions]

Results for /rerun-test test/registered/eval/test_vlms_mmmu_eval.py:

🚀 2-gpu-h100 (1 test): ✅ View workflow run

cd test/ && python3 registered/eval/test_vlms_mmmu_eval.py