Skip to content

[Bugfix][CI] Set cudagraph_mode=FULL for the Ernie4.5-VL ViT cudagraph test - #54957

Merged
AndreasKaratzas merged 2 commits into
vllm-project:mainfrom
stefankoncarevic:ernie-vl-cudagraph-mode
Sep 2, 2026
Merged

[Bugfix][CI] Set cudagraph_mode=FULL for the Ernie4.5-VL ViT cudagraph test#54957
AndreasKaratzas merged 2 commits into
vllm-project:mainfrom
stefankoncarevic:ernie-vl-cudagraph-mode

Conversation

@stefankoncarevic

@stefankoncarevic stefankoncarevic commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Ernie4.5-VL ViT cudagraph test fails after #54782

Failing test: test_vit_cudagraph_image[ernie45_vl]

AMD CI: https://buildkite.com/vllm/amd-ci/builds/12553/list?sid=01a0615a-019d-4492-a92c-a4018dfdcc8b&tab=output

The same group failed on both MI300 and MI355.

Error

RuntimeError: Ernie4_5_VLMoeForConditionalGeneration: piecewise CUDA graphs
(cudagraph_mode=FULL_AND_PIECEWISE) unavailable, model is not torch-compiled
and breakable CUDA graph is off. Set VLLM_USE_BREAKABLE_CUDAGRAPH=1 or
cudagraph_mode=NONE/FULL.

Raised in vllm/v1/worker/gpu/cudagraph_utils.py, during engine start.

What exposed it

1c26e57d3c — [Bugfix] Raise for unavailable piecewise CUDA graphs (#54782).
It added a guard that raises when piecewise cudagraphs are requested for a
model with no active @support_torch_compile wrapper.

Root cause

The guard is correct; the misconfiguration predates it. Ernie4_5_VLMoeModel
keeps its @support_torch_compile decorator commented out, since the model
was added in #22514:

# Since Ernie VL distinguishes between text experts and vision experts,
# enabling torch.compile will cause errors.
# @support_torch_compile(

So the model is not compiled, piecewise cudagraphs have nothing to partition,
and the test asks for the default FULL_AND_PIECEWISE. Until #54782 this
silently captured nothing.

Not platform-specific: VLLM_USE_BREAKABLE_CUDAGRAPH defaults to False
everywhere and the test is gated on is_cuda_alike(), so CUDA should hit it
too.

Ernie is the only affected model in that file — glm4_1v and step3_vl pass,
and deepseek_ocr already pins cudagraph_mode.

Fix

Pin the mode for the Ernie entry, the way deepseek_ocr in the same file
already does:

compilation_config_overrides={
    "cudagraph_mode": 2,  # FULL
},

Verified on MI300X: fails in 31s before, passes in 40s after.


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

…h test

Ernie4_5_VLMoeModel keeps @support_torch_compile commented out, because
its split of text and vision experts breaks compilation, so piecewise
cudagraphs have nothing to partition. The test asked for the default
FULL_AND_PIECEWISE, which silently captured nothing until the guard
added in vllm-project#54782 started raising for it.

Pin the mode to FULL, as the deepseek_ocr entry in the same file already
does. Only the encoder graphs this test covers are captured.

Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@mergify mergify Bot added multi-modality Related to multi-modality (#4194) nvidia bug Something isn't working labels Sep 2, 2026
@AndreasKaratzas

Copy link
Copy Markdown
Member

/ci run

@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown

✅ Triggered Buildkite CI #86900 for commit cba90d1913dd.

@github-project-automation github-project-automation Bot moved this to Ready in NVIDIA Sep 2, 2026
@AndreasKaratzas
AndreasKaratzas merged commit 2691c6c into vllm-project:main Sep 2, 2026
34 checks passed
@github-project-automation github-project-automation Bot moved this from Ready to Done in NVIDIA Sep 2, 2026
mylibrar pushed a commit to tanyuqian/vllm that referenced this pull request Sep 3, 2026
…h test (vllm-project#54957)

Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working multi-modality Related to multi-modality (#4194) nvidia

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants