[Model] Batch MiniCPM-o 4.5 cold reference CFM setup

docs/design/minicpm_o45_mrv2_performance.md

@@ -64,10 +64,27 @@ preserving CFM step-graph and TRT dispatch. Mixed-history arena replay is
 restricted to CUDA; the existing opt-in slot path is unchanged on other
 platforms.
 
-Reference extraction and initial cache setup remain separate for each
-reference; speaker conditioning, attention lengths and cache trimming stay
+Reference extraction, encoder initialization and speaker projection remain
+per reference. With FP32/TF32 CUDA fused ragged Whole-Euler and zero CFM frame padding,
+compatible cold references share an initial CFM solve, bounded by the configured
+microbatch and graph batch limits. Setup-cache hits are reused, and each
+reference retains its own cached state. A refused graph replay retries setup
+per reference using the already-computed encoder and speaker-projection outputs,
+preserving the existing solver selection. FP16, TF32 opt-out,
+unfused bodies (including BF16 attention caches) and other execution
+configurations keep per-reference initialization. Speaker
+conditioning, attention lengths and cache trimming stay
 per request. Cache epochs remain separate unless cross-turn batching is enabled.
 
+Cold setup shares the `cfm_max_graphs` budget with decode graphs. For eligible
+batched setup, non-slot precapture warms the default prompt's offset-zero graphs
+after decode shapes, using remaining budget with M=1 first. Other reference
+widths capture lazily or fall back to per-reference solves. During batched setup,
+the stacked CFG attention output and independent per-reference cache copies
+coexist, increasing transient memory with the group size. The microbatch bound
+limits this peak; independent copies keep one reference from retaining another
+reference's buffers after cache eviction.
+
 The MiniCPM Code2Wav stage runs CAM++ speaker embedding on CUDA and NPU using
 the in-tree torch model with FP32 weights and inputs, loaded from the
 checkpoint's `campplus.onnx`. On CUDA, reference preparation follows the
@@ -77,8 +94,8 @@ strict FP32 arithmetic.
 The CUDA path does not require TensorRT. Set `token2wav_device_spk: false`
 in connector `extra` to use CPU ONNX Runtime for accuracy triage; explicitly
 enabling `token2wav_trt` retains its TensorRT speaker embedding path.
-This moves speaker embedding off the CPU but does not batch reference
-extraction or initial cache setup. Device kernels can change floating-point
+This moves speaker embedding off the CPU; reference extraction stays separate
+from the initial CFM batching described above. Device kernels can change floating-point
 rounding, so this does not promise bit-identical embeddings or waveforms.
 
 The ragged Whole-Euler path masks padded history and returns each request's

tests/model_executor/models/minicpmo_4_5/test_cfm_graph_bucketing.py

@@ -305,8 +305,9 @@ def test_zero_padded_cnn_cache_uniform_blocks_use_one_write():
     assert torch.count_nonzero(cache[..., 3:]) == 0
 
 
-@pytest.mark.parametrize("max_graphs", [15, 32])
-def test_precapture_covers_final_chunk_without_displacing_steady_graphs(monkeypatch, max_graphs):
+@pytest.mark.parametrize("max_graphs", [15, 30, 31, 35])
+@pytest.mark.parametrize("setup_frames", [0, 273])
+def test_precapture_covers_final_chunk_without_displacing_steady_graphs(monkeypatch, max_graphs, setup_frames):
     from vllm_omni.model_executor.models.minicpmo_4_5 import cuda_graph_wrapper as graph_module
 
     captured = []
@@ -336,14 +337,24 @@ def entry(*, graph_batch, query_cap, offset, **kwargs):
     )
     monkeypatch.setattr(graph_module, "_memory_snapshot", lambda device: None)
     count = WholeEulerCFMGraphWrapper.precapture(
-        wrapper, offsets=range(300, 401), steady=400, channels=4, spk_dim=4, tail_frames=6
+        wrapper,
+        offsets=range(300, 401),
+        steady=400,
+        channels=4,
+        spk_dim=4,
+        tail_frames=6,
+        setup_frames=setup_frames,
     )
     assert captured[:15] == [(b, 50, o) for b in [16, 8, 4, 2, 1] for o in [400, 350, 300]]
-    assert count == min(max_graphs, 30)
+    expected = [(b, 50, o) for b in [16, 8, 4, 2, 1] for o in [400, 350, 300]]
+    expected += [(b, 100, o) for b in [16, 8, 4, 2, 1] for o in [400, 350, 300]]
+    if setup_frames:
+        expected += [(b, 300, 0) for b in [1, 2, 4, 8, 16]]
+    assert captured == expected[:max_graphs]
+    assert count == len(captured)
     assert wrapper.query_widths == (50,)
-    if max_graphs == 32:
+    if max_graphs >= 30:
         assert (1, 100, 350) in captured
-        assert captured[15:] == [(b, 100, o) for b in [16, 8, 4, 2, 1] for o in [400, 350, 300]]
 
 
 @pytest.mark.parametrize("width,valid", [(0, 0), (3, 2), (8, 8)])