Revert "[MM][CG] Support ViT full CUDA graph for Ernie-4.5-VL image inference" (#45254) - #51263
Revert "[MM][CG] Support ViT full CUDA graph for Ernie-4.5-VL image inference" (#45254)#51263vllm-agent wants to merge 1 commit into
Conversation
…nference (vllm-project#45254)" This reverts commit 777b01d.
|
Documentation preview: https://vllm--51263.org.readthedocs.build/en/51263/ |
…th outputs `postprocess_encoder_output` now receives `outputs: dict[str, torch.Tensor]` keyed by encoder path, but Ernie's override still treated it as a single tensor, so `output.device` raised `AttributeError: 'dict' object has no attribute 'device'` during encoder CUDA graph replay. Extract the single "default" path output before use. This is the failure that prompted the revert in vllm-project#51263; fix-forward instead. Signed-off-by: Qiuyang Yue <yueqiuyang1389@gmail.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Thanks for catching this It's a semantic merge conflict, not a bug in the original logic. The SupportsEncoderCudaGraph.postprocess_encoder_output signature changed to a multi-path form — it now receives outputs: dict[str, torch.Tensor] keyed by encoder path, and the caller in encoder_cudagraph.py passes graph_outputs (a dict). #45254 was written and CI-green against the older single-tensor signature; the two landed close together, so each was green independently but the combination broke Ernie's override: I've opened a one-line fix in #51461 — it just updates Ernie's override to the current signature and pulls out the single "default" path output: Verified locally with the exact case from build 82629: Could we merge #51461 and close this revert PR? Thank you! |
|
superseded by #51461 |
Reverts #45254 — [MM][CG] Support ViT full CUDA graph for Ernie-4.5-VL image inference
Why: the new
Ernie4_5_VLMoeForConditionalGeneration.postprocess_encoder_outputcrashes on its own test case.outputis a dict at this call site, so encoder CUDA-graph replay fails for Ernie-4.5-VL immediately.Multi-Modal Models (Extended Generation 1)—tests/models/multimodal/generation/test_vit_cudagraph.py::test_vit_cudagraph_image[ernie45_vl]ernie45_vlcase was added by this same PR, so the job was green before it merged.The revert applied cleanly with no conflicts. Happy to close this in favour of a one-line fix-forward.
Auto-generated by CI failure analyzer.