Skip to content

[Bugfix] DiffusionGemma: fix multimodal support - #57589

Merged
LucasWilkinson merged 4 commits into
vllm-project:mainfrom
mmastrac:dgemma-mm-lora-flag
Sep 21, 2026
Merged

LucasWilkinson merged 4 commits into
vllm-project:mainfrom
mmastrac:dgemma-mm-lora-flag

Conversation

@mmastrac

@mmastrac mmastrac commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

Purpose

DiffusionGemma reuses Gemma4ForConditionalGeneration._process_image_input for its image path. That method reads self._enable_mm_lora, which Gemma4's constructor sets. DiffusionGemma's doesn't, so any request with an image raises AttributeError in the encoder runner and takes the engine down.

Set the flag in DiffusionGemma's constructor the same way Gemma4 does.

Required for #57250 to support multimodal inputs.

Test plan

Hardware: 1 DGX Spark.

Send an image.

vllm serve nvidia/diffusiongemma-26B-A4B-it-NVFP4 --max-model-len 4096 --diffusion-config '{"canvas_length": 64}'
curl -s localhost:8000/v1/chat/completions -H 'content-type: application/json' -d '{
  "model": "nvidia/diffusiongemma-26B-A4B-it-NVFP4", "max_tokens": 32,
  "messages": [{"role": "user", "content": [
    {"type": "image_url", "image_url": {"url": "https://upload.wikimedia.org/wikipedia/commons/thumb/4/47/PNG_transparency_demonstration_1.png/280px-PNG_transparency_demonstration_1.png"}},
    {"type": "text", "text": "What is in the image?"}]}]}'

Test results

Without patch: 500 error (AttributeError)

With patch: a successful response


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Gemma4ForConditionalGeneration._process_image_input, which this model
reuses, reads self._enable_mm_lora, set in Gemma4's constructor and not in
this one, so any image request raised AttributeError in the encoder runner
and took the engine down.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Matt Mastracci <matthew@mastracci.com>

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@mergify mergify Bot added the bug Something isn't working label Sep 18, 2026

@LucasWilkinson LucasWilkinson left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thanks for splitting this out!

@LucasWilkinson

Copy link
Copy Markdown
Contributor

/ci run

@LucasWilkinson

Copy link
Copy Markdown
Contributor

/ci run

@LucasWilkinson LucasWilkinson added the ready ONLY add when PR is ready to merge/full CI is needed label Sep 18, 2026
@github-actions

Copy link
Copy Markdown

✅ Triggered Buildkite CI #89902 for commit 55e28de3232d.

@github-actions

Copy link
Copy Markdown

✅ @mmastrac, CI is now available for this PR.

  • /ci run starts upstream CI; /amd-ci run starts AMD CI only.
  • Your branch must contain every commit currently on its upstream target branch. Merge or rebase onto the latest target branch, then rerun the command. Append --allow-stale to a run command to test an outdated branch at your own risk.
  • /ci retry retries failed jobs in the CI build for the current PR head. If the current head has no CI build, it starts a new CI build for the current head containing only jobs that failed in the latest earlier CI build for this PR.
  • /amd-ci retry retries failed jobs in AMD CI for the current PR head. Use /amd-ci run when the current head has no AMD CI build.
  • /ci cancel cancels scheduled or running CI builds for this PR branch; /amd-ci cancel does the same for AMD CI only.

@github-actions

Copy link
Copy Markdown

✅ CI is already running for this commit: https://buildkite.com/vllm/ci/builds/89902

@mmastrac

Copy link
Copy Markdown
Contributor Author

/ci run

@github-actions

Copy link
Copy Markdown

✅ Triggered Buildkite CI #90083 for commit 73e6401b8670.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants