fix(vllm): declare entry-stage engine_input_source in GLM-Image NIXL config - #12709
Merged
Merged
Conversation
…config The AR stage (stage 0) of the GLM-Image AR->DiT NIXL pipeline fails to start on vllm-omni v0.26.0rc1: RuntimeError: Orchestrator initialization failed: Missing key engine_input_source (full_key: stage_args[0].engine_input_source) vllm-omni dropped the getattr default for this field in v0.26.0rc1, so stage metadata extraction now reads it unconditionally. The legacy stage_args config format applies no schema defaults, so stage 0 — which omitted the field as the entry stage — no longer resolves. Stage 1 already declares engine_input_source, which is why only the AR half crashed. Declare it explicitly as an empty list, matching what the deploy-config parser supplies for an entry stage. Refs: DYN-3746 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (1)
WalkthroughThe AR entry stage configuration now explicitly declares an empty ChangesAR entry configuration
Estimated code review effort: 1 (Trivial) | ~2 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Comment |
rmccorm4
approved these changes
Aug 5, 2026
GuanLuo
enabled auto-merge (squash)
August 5, 2026 18:28
This comment has been minimized.
This comment has been minimized.
ayushag-nv
approved these changes
Aug 5, 2026
2 tasks
hhzhang16
added a commit
that referenced
this pull request
Aug 6, 2026
dyn-3691-extract-shared-target-pid-cuda-customstorage-operation-layer * 'main' of https://github.com/ai-dynamo/dynamo: (65 commits) fix(frontend): emit SGLang stream role once (#12741) docs(fern): promote v1.3.1 to current release (#12752) fix(docs): remove duplicate unscoped community-rail CSS rules (#12615) feat(operator): migrate CRD storage to v1beta1 (#11904) fix: synchronize self-benchmark capacity across DP ranks (#12021) chore(deps): bump dynamo-tokenizers to 1.8.0 (#12707) fix(frontend): preserve split UTF-8 characters (#12688) docs: align Kubernetes build selector with CLI (#12729) fix(frontend): preserve completion backend error status (#12706) fix(operator): replace snapshot pods after GMS restart (#11286) refactor(media): rename installer module, drop --packages per review fix(media): harden installer against three pre-redesign review findings fix(media): verify installs in a fresh interpreter; teach --pip-args= form test(serve): install test-time decoders at the validated bounds feat(media): explicit installer for additional media decoders docs(spica): correct kv_load_ratio support guidance (#12714) feat(operator): add experimental grove.forceScalingGroup for single-node components (#11772) fix(vllm): declare entry-stage engine_input_source in GLM-Image NIXL config (#12709) chore: bump trtllm to v1.3.0rc23 (#12532) perf: remove trtllm postprocessing workers from the args as post processing workers are not effective in dynamo (#12592) ... Signed-off-by: Hannah Zhang <hannahz@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview:
The GLM-Image AR→DiT NIXL disaggregation launch script
(
examples/backends/vllm/launch/disagg_omni_glm_image_nixl.sh) cannot start onvllm-omni v0.26.0rc1. The AR stage exits with code 1 and the frontend never
becomes ready:
Summary
Stage 0 of
stage_configs/glm_image_nixl.yamlis the pipeline's entry stage andomitted
engine_input_source. That was valid through v0.25.0rc1, wherevllm-omni defaulted the field to
[]. Upstream removed the default inv0.26.0rc1 (
fb4dd41d, "[Refactor]: Remove unnecessary config getattr" #5199),so stage metadata extraction now reads it unconditionally. The legacy
stage_argsconfig format applies no schema defaults, so the field is simplyabsent and the orchestrator thread dies during engine creation.
Stage 1 already declares
engine_input_source: [0], which is why only the ARhalf crashed. The SHM variant is unaffected because it consumes vllm-omni's own
packaged config in the
stagesformat, whose parser supplies the default.This declares the field explicitly as an empty list — the value upstream itself
supplied until v0.26.0rc1, and the one its deploy parser still produces for an
entry stage (a stage with empty
input_sources).Details:
Fixing the config rather than
_stage_config_to_dictinstage_worker.py:that file ships in the wheel and is on the path of every omni deployment,
including the SHM path that currently works, whereas this change touches one
example config. Re-adding in Dynamo a default upstream deliberately removed
would also diverge from the engine's contract.
Where should the reviewer start?
Single file, four added lines:
examples/backends/vllm/launch/stage_configs/glm_image_nixl.yamlValidation
v0.26.0rc1checkout, matching thereported
ConfigAttributeErrorexactly includingfull_keyandobject_type; confirmed it no longer raises after this change.single-stage config that is handed to
AsyncOmni.pre-commit run --files examples/backends/vllm/launch/stage_configs/glm_image_nixl.yamlpasses.host. This config has no CI lane, and the AR stage previously died before any
transfer occurred, so the NIXL data path has not executed on this pin.
Related Issues
🚫 This PR is NOT linked to an issue:
🤖 Generated with Claude Code
Summary by CodeRabbit