Skip to content

[Bugfix] Fix Gemma3n concurrent audio requests crashing EngineCore - #38305

Closed
he-yufeng wants to merge 1 commit into
vllm-project:mainfrom
he-yufeng:fix/gemma3n-audio-dynamic-dims
Closed

he-yufeng wants to merge 1 commit into
vllm-project:mainfrom
he-yufeng:fix/gemma3n-audio-dynamic-dims

Conversation

@he-yufeng

Copy link
Copy Markdown
Contributor

Summary

Gemma3nAudioInputs declares input_features_padded and input_features_mask with fixed sequence-length dimensions. When two concurrent requests carry audio clips of different durations, TensorSchema.validate() sees mismatched shapes and kills EngineCore:

ValueError: input_features_padded contains inconsistent shapes:
  torch.Size([496, 128]) (index 0) vs torch.Size([449, 128]) (index 1)

The fix adds dynamic_dims={"s"} to both fields, matching the pattern established by PR #31223 for MiniCPM-o, Qwen2Audio, and AudioFlamingo3.

Changes

  • vllm/model_executor/models/gemma3n_mm.py: Add dynamic_dims={"s"} to input_features_padded and input_features_mask in Gemma3nAudioInputs

Test Plan

  • Verify concurrent audio requests to Gemma3n no longer crash
  • Existing multimodal tests pass

Fixes #38297

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the Gemma3nAudioInputs schema in vllm/model_executor/models/gemma3n_mm.py to explicitly mark the sequence dimension 's' as dynamic for both input_features_padded and input_features_mask. I have no feedback to provide.

@mergify mergify Bot added the bug Something isn't working label Mar 27, 2026
@he-yufeng
he-yufeng force-pushed the fix/gemma3n-audio-dynamic-dims branch from e470c66 to 24a1039 Compare March 30, 2026 13:22
@he-yufeng

Copy link
Copy Markdown
Contributor Author

bump — 2-line fix, same pattern as #31223.

@ereish64

Copy link
Copy Markdown

I was having the same issue. Thanks for the PR.

@he-yufeng

Copy link
Copy Markdown
Contributor Author

Rebased this onto current main to rerun the stale pre-run gate. The previous failure was from 2026-03-30 and required either a ready label or 4+ merged PRs; the author now has 5 merged vLLM PRs, so the gate should pass on the refreshed run. Local checks: py_compile on vllm/model_executor/models/gemma3n_mm.py and git diff --check.

@he-yufeng
he-yufeng force-pushed the fix/gemma3n-audio-dynamic-dims branch from 24a1039 to 77afc38 Compare May 13, 2026 21:04
Gemma3nAudioInputs declares input_features_padded and
input_features_mask with fixed sequence-length dimensions. When two
concurrent requests carry audio clips of different durations,
TensorSchema.validate() sees mismatched shapes and kills EngineCore
with a ValueError.

Mark the sequence-length dimension as dynamic so the validator allows
variable-length audio inputs in the same batch, matching the pattern
used by MiniCPM-o, Qwen2Audio, and AudioFlamingo3 (PR vllm-project#31223).

Fixes vllm-project#38297

Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
@he-yufeng
he-yufeng force-pushed the fix/gemma3n-audio-dynamic-dims branch from 77afc38 to 02c44f2 Compare May 21, 2026 05:32
@he-yufeng

Copy link
Copy Markdown
Contributor Author

Rebased this branch onto current main and re-ran the lightweight checks for the Gemma3n audio schema change.

Validation:

python -m ruff check vllm\model_executor\models\gemma3n_mm.py
python -m ruff format --check vllm\model_executor\models\gemma3n_mm.py
python -m py_compile vllm\model_executor\models\gemma3n_mm.py
git diff --check

@he-yufeng

Copy link
Copy Markdown
Contributor Author

Closing this during PR backlog cleanup. This one has not received human maintainer signal and is lower priority than the remaining vLLM PRs that are approved, actively reviewed, or more recent/high-signal. I can reopen or refresh it if maintainers want this specific fix.

@he-yufeng he-yufeng closed this Jun 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Gemma3n concurrent audio requests crash EngineCore — missing dynamic_dims on audio sequence dimension

2 participants