Repository navigation
[Config] Nest model-specific diffusion options under DiffusionConfig.model_options - #2
Open
LucasWilkinson wants to merge 2 commits into
Conversation
Keep DiffusionConfig limited to options shared by all diffusion models. Architectures declare the DiffusionConfig subclass that --diffusion-config is parsed into through their VerifyAndUpdateConfig hook, so Nemotron-only options are validated for Nemotron and rejected for other models. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
DiffusionConfig parses and documents itself without a model. Model-specific options live under model_options and are validated by each model's VerifyAndUpdateConfig hook (NemotronDiffusionOptions, DiffusionGemmaDiffusionOptions). DiffusionGemma options override the generation_config.json values, and its hook now backfills canvas_length when --diffusion-config omits it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
LucasWilkinson
changed the title
5 of 6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Suggestion for vllm-project#58387.
DiffusionConfigkeeps only the options every diffusion model shares (canvas_length,max_denoising_steps), plus a documentedmodel_optionsfield. Model-specific options go undermodel_options, and each model'sVerifyAndUpdateConfighook validates them against its own options class:NemotronDiffusionOptions:temperature,selection_policy,confidence_threshold, moved fromDiffusionConfig.DiffusionGemmaDiffusionOptions:t_min,t_max,entropy_bound,confidence_threshold,stability_threshold. These were previously read only fromgeneration_config.json, which they now override.DiffusionConfigparses and documents itself without a model, and each model rejects keys that belong to another model. Example:DiffusionGemma's hook now also backfills
canvas_lengthwhen--diffusion-configomits it, as Nemotron's hook already does.Tests:
pytest tests/models/language/generation/test_nemotron_dllm.py tests/test_sampling_params.py tests/v1/sample/test_diffusion_gemma_reads.py: 105 passed, 5 skipped.nvidia/Nemotron-Labs-Diffusion-3Bwithmodel_options={"temperature": 0.0, "selection_policy": "leftmost"}: options parsed intoNemotronDiffusionOptions; greedy output correct;{"t_min": 0.1}rejected. Run withTORCH_COMPILE_DISABLE=1because of the B300 compile error noted in the review.google/diffusiongemma-26B-A4B-itwithmodel_options={"stability_threshold": 2}: options parsed intoDiffusionGemmaDiffusionOptions,canvas_lengthbackfilled to 256, generation runs;{"temperature": 0.0}rejected.AI assistance (Claude) was used to write this change.
🤖 Generated with Claude Code