Repository navigation
[Bugfix] DiffusionGemma: cast the self-conditioning soft embed to the buffer dtype - #57462
Merged
LucasWilkinson merged 3 commits intoSep 20, 2026
Merged
Conversation
… buffer dtype The soft embed is computed in the embedding weight's dtype and written into the fp32 self-conditioning buffer. Inductor inserts the cast, so the compiled step is fine, but once dynamo falls back to eager (past the recompile limit, for instance when requests come in several canvas widths) the index put raises a dtype mismatch and the engine dies. Cast explicitly so both paths agree. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Matt Mastracci <matthew@mastracci.com>
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
4 tasks done
LucasWilkinson
approved these changes
Sep 18, 2026
LucasWilkinson
added
the
ready
Contributor
|
/ci run |
|
✅ @mmastrac, CI is now available for this PR.
|
Contributor
|
/ci run |
LucasWilkinson
enabled auto-merge (squash)
September 18, 2026 21:28
|
✅ Triggered Buildkite CI #89919 for commit |
|
✅ CI is already running for this commit: https://buildkite.com/vllm/ci/builds/89919 |
Contributor
Author
|
/ci retry |
|
✅ Queued 2 failed job(s) for retry in Buildkite CI #89919. |
Contributor
Author
|
/ci retry |
|
✅ Queued 2 failed job(s) for retry in Buildkite CI #89919. |
Contributor
Author
|
/ci retry |
|
✅ Queued 2 failed job(s) for retry in Buildkite CI #89919. |
Contributor
Author
|
/ci run |
|
✅ Triggered Buildkite CI #90003 for commit |
Contributor
Author
|
/ci retry |
|
✅ Queued 1 failed job(s) for retry in Buildkite CI #90003. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
The self-conditioning soft embed is computed in the embedding weight's dtype and written
into the fp32
self_conditioning_embedsbuffer.Once dynamo falls back to eager (past the recompile limit, which happens when requests
arrive in several canvas widths) the index put raises a dtype mismatch and the engine dies.
Cast explicitly so the eager and compiled paths agree.
Test plan
Hardware: 1 DGX Spark,
google/diffusiongemma-26B-A4B-it.Setting
TORCHDYNAMO_DISABLE=1disablestorch.compileentirely, so_compiled_sample_stepruns eagerly from the very first denoise step and hits the dtype-mismatched index put immediatelyTORCHDYNAMO_DISABLE=1 vllm serve google/diffusiongemma-26B-A4B-itEssential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.