Skip to content

Revert "[MM][CG] Support ViT full CUDA graph for Ernie-4.5-VL image inference" (#45254) - #51263

Closed
vllm-agent wants to merge 1 commit into
vllm-project:mainfrom
vllm-agent:auto-revert/pr-45254
Closed

Revert "[MM][CG] Support ViT full CUDA graph for Ernie-4.5-VL image inference" (#45254)#51263
vllm-agent wants to merge 1 commit into
vllm-project:mainfrom
vllm-agent:auto-revert/pr-45254

Conversation

@vllm-agent

Copy link
Copy Markdown
Contributor

Reverts #45254 — [MM][CG] Support ViT full CUDA graph for Ernie-4.5-VL image inference

Why: the new Ernie4_5_VLMoeForConditionalGeneration.postprocess_encoder_output crashes on its own test case.

File "vllm/model_executor/models/ernie45_vl.py", line 1734, in postprocess_encoder_output
    grid_thw = batch_mm_kwargs["image_grid_thw"].to(output.device)
AttributeError: 'dict' object has no attribute 'device'

output is a dict at this call site, so encoder CUDA-graph replay fails for Ernie-4.5-VL immediately.

  • Failing job: Multi-Modal Models (Extended Generation 1)tests/models/multimodal/generation/test_vit_cudagraph.py::test_vit_cudagraph_image[ernie45_vl]
  • Failures linked to this PR: 1
  • Build: https://buildkite.com/vllm/ci/builds/82629 (commit c0202c5)
  • The ernie45_vl case was added by this same PR, so the job was green before it merged.

The revert applied cleanly with no conflicts. Happy to close this in favour of a one-line fix-forward.

Auto-generated by CI failure analyzer.

@mergify

mergify Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Documentation preview: https://vllm--51263.org.readthedocs.build/en/51263/

@mergify mergify Bot added documentation Improvements or additions to documentation multi-modality Related to multi-modality (#4194) nvidia labels Aug 6, 2026
qyYue1389 added a commit to qyYue1389/vllm that referenced this pull request Aug 7, 2026
…th outputs

`postprocess_encoder_output` now receives `outputs: dict[str, torch.Tensor]`
keyed by encoder path, but Ernie's override still treated it as a single
tensor, so `output.device` raised `AttributeError: 'dict' object has no
attribute 'device'` during encoder CUDA graph replay. Extract the single
"default" path output before use.

This is the failure that prompted the revert in vllm-project#51263; fix-forward instead.

Signed-off-by: Qiuyang Yue <yueqiuyang1389@gmail.com>

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@qyYue1389

qyYue1389 commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Thanks for catching this

It's a semantic merge conflict, not a bug in the original logic. The SupportsEncoderCudaGraph.postprocess_encoder_output signature changed to a multi-path form — it now receives outputs: dict[str, torch.Tensor] keyed by encoder path, and the caller in encoder_cudagraph.py passes graph_outputs (a dict). #45254 was written and CI-green against the older single-tensor signature; the two landed close together, so each was green independently but the combination broke Ernie's override:

grid_thw = batch_mm_kwargs["image_grid_thw"].to(output.device)
AttributeError: 'dict' object has no attribute 'device'

I've opened a one-line fix in #51461 — it just updates Ernie's override to the current signature and pulls out the single "default" path output:

def postprocess_encoder_output(self, outputs: dict[str, torch.Tensor], ...):
    output = outputs["default"]
    grid_thw = batch_mm_kwargs["image_grid_thw"].to(output.device)
    ...

Verified locally with the exact case from build 82629:
pytest tests/models/multimodal/generation/test_vit_cudagraph.py::test_vit_cudagraph_image[ernie45_vl]

Could we merge #51461 and close this revert PR? Thank you!

@khluu

khluu commented Aug 10, 2026

Copy link
Copy Markdown
Member

superseded by #51461

@khluu khluu closed this Aug 10, 2026
@github-project-automation github-project-automation Bot moved this to Done in NVIDIA Aug 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation multi-modality Related to multi-modality (#4194) nvidia

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants