Skip to content

fix(vllm): declare entry-stage engine_input_source in GLM-Image NIXL config - #12709

Merged
GuanLuo merged 1 commit into
mainfrom
gluo/dyn-3746-omni-nixl-entry-stage-input-source
Aug 5, 2026
Merged

fix(vllm): declare entry-stage engine_input_source in GLM-Image NIXL config#12709
GuanLuo merged 1 commit into
mainfrom
gluo/dyn-3746-omni-nixl-entry-stage-input-source

Conversation

@GuanLuo

@GuanLuo GuanLuo commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Overview:

The GLM-Image AR→DiT NIXL disaggregation launch script
(examples/backends/vllm/launch/disagg_omni_glm_image_nixl.sh) cannot start on
vllm-omni v0.26.0rc1. The AR stage exits with code 1 and the frontend never
becomes ready:

RuntimeError: Orchestrator initialization failed: Missing key
engine_input_source (full_key: stage_args[0].engine_input_source)

Summary

Stage 0 of stage_configs/glm_image_nixl.yaml is the pipeline's entry stage and
omitted engine_input_source. That was valid through v0.25.0rc1, where
vllm-omni defaulted the field to []. Upstream removed the default in
v0.26.0rc1 (fb4dd41d, "[Refactor]: Remove unnecessary config getattr" #5199),
so stage metadata extraction now reads it unconditionally. The legacy
stage_args config format applies no schema defaults, so the field is simply
absent and the orchestrator thread dies during engine creation.

Stage 1 already declares engine_input_source: [0], which is why only the AR
half crashed. The SHM variant is unaffected because it consumes vllm-omni's own
packaged config in the stages format, whose parser supplies the default.

This declares the field explicitly as an empty list — the value upstream itself
supplied until v0.26.0rc1, and the one its deploy parser still produces for an
entry stage (a stage with empty input_sources).

Details:

Fixing the config rather than _stage_config_to_dict in stage_worker.py:
that file ships in the wheel and is on the path of every omni deployment,
including the SHM path that currently works, whereas this change touches one
example config. Re-adding in Dynamo a default upstream deliberately removed
would also diverge from the engine's contract.

Where should the reviewer start?

Single file, four added lines:
examples/backends/vllm/launch/stage_configs/glm_image_nixl.yaml

Validation

  • Reproduced the original failure against a v0.26.0rc1 checkout, matching the
    reported ConfigAttributeError exactly including full_key and
    object_type; confirmed it no longer raises after this change.
  • Verified the field survives Dynamo's whitelist copy into the synthesized
    single-stage config that is handed to AsyncOmni.
  • pre-commit run --files examples/backends/vllm/launch/stage_configs/glm_image_nixl.yaml passes.
  • Not yet validated: end-to-end image generation over NIXL on a two-GPU
    host. This config has no CI lane, and the AR stage previously died before any
    transfer occurred, so the NIXL data path has not executed on this pin.

Related Issues

🚫 This PR is NOT linked to an issue:

  • Confirmed — no related issue

🤖 Generated with Claude Code


Open in Devin Review

Summary by CodeRabbit

  • Bug Fixes
    • Updated the GLM image stage configuration to explicitly define an empty engine input source, improving configuration clarity and consistency.

…config

The AR stage (stage 0) of the GLM-Image AR->DiT NIXL pipeline fails to start
on vllm-omni v0.26.0rc1:

  RuntimeError: Orchestrator initialization failed: Missing key
  engine_input_source (full_key: stage_args[0].engine_input_source)

vllm-omni dropped the getattr default for this field in v0.26.0rc1, so stage
metadata extraction now reads it unconditionally. The legacy stage_args config
format applies no schema defaults, so stage 0 — which omitted the field as the
entry stage — no longer resolves. Stage 1 already declares engine_input_source,
which is why only the AR half crashed.

Declare it explicitly as an empty list, matching what the deploy-config parser
supplies for an entry stage.

Refs: DYN-3746

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>
@GuanLuo
GuanLuo requested a review from a team as a code owner August 5, 2026 18:23
@github-actions github-actions Bot added fix backend::vllm Relates to the vllm backend labels Aug 5, 2026

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 55c726e5-231c-4367-aa36-4a8d774c5542

📥 Commits

Reviewing files that changed from the base of the PR and between e322c8d and ec83ccf.

📒 Files selected for processing (1)
  • examples/backends/vllm/launch/stage_configs/glm_image_nixl.yaml

Walkthrough

The AR entry stage configuration now explicitly declares an empty engine_input_source instead of relying on schema defaults.

Changes

AR entry configuration

Layer / File(s) Summary
Explicit engine input source
examples/backends/vllm/launch/stage_configs/glm_image_nixl.yaml
The AR entry stage declares an empty engine_input_source.

Estimated code review effort: 1 (Trivial) | ~2 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely identifies the configuration fix for the GLM-Image NIXL entry stage.
Description check ✅ Passed The description covers the required sections, explains the failure and fix, identifies the review location, and confirms no related issue.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

@GuanLuo
GuanLuo enabled auto-merge (squash) August 5, 2026 18:28
@datadog-official

This comment has been minimized.

@GuanLuo
GuanLuo merged commit 0c4fabc into main Aug 5, 2026
113 checks passed
@GuanLuo
GuanLuo deleted the gluo/dyn-3746-omni-nixl-entry-stage-input-source branch August 5, 2026 21:07
pvijayakrish pushed a commit that referenced this pull request Aug 5, 2026
…config (cherry-pick #12709) (#12719)

Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
hhzhang16 added a commit that referenced this pull request Aug 6, 2026
dyn-3691-extract-shared-target-pid-cuda-customstorage-operation-layer

* 'main' of https://github.com/ai-dynamo/dynamo: (65 commits)
  fix(frontend): emit SGLang stream role once (#12741)
  docs(fern): promote v1.3.1 to current release (#12752)
  fix(docs): remove duplicate unscoped community-rail CSS rules (#12615)
  feat(operator): migrate CRD storage to v1beta1 (#11904)
  fix: synchronize self-benchmark capacity across DP ranks (#12021)
  chore(deps): bump dynamo-tokenizers to 1.8.0 (#12707)
  fix(frontend): preserve split UTF-8 characters (#12688)
  docs: align Kubernetes build selector with CLI (#12729)
  fix(frontend): preserve completion backend error status (#12706)
  fix(operator): replace snapshot pods after GMS restart (#11286)
  refactor(media): rename installer module, drop --packages per review
  fix(media): harden installer against three pre-redesign review findings
  fix(media): verify installs in a fresh interpreter; teach --pip-args= form
  test(serve): install test-time decoders at the validated bounds
  feat(media): explicit installer for additional media decoders
  docs(spica): correct kv_load_ratio support guidance (#12714)
  feat(operator): add experimental grove.forceScalingGroup for single-node components (#11772)
  fix(vllm): declare entry-stage engine_input_source in GLM-Image NIXL config (#12709)
  chore: bump trtllm to v1.3.0rc23 (#12532)
  perf: remove trtllm postprocessing workers from the args as post processing workers are not effective in dynamo (#12592)
  ...

Signed-off-by: Hannah Zhang <hannahz@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend::vllm Relates to the vllm backend fix size/XS

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants