Skip to content

fix(vllm): migrate GLM-Image NIXL config to the stages deploy schema (cherry-pick #12984) - #13033

Merged
pvijayakrish merged 1 commit into
release/1.4.0from
gluo/cherrypick-12984-glm-image-nixl-stages-schema
Aug 11, 2026
Merged

fix(vllm): migrate GLM-Image NIXL config to the stages deploy schema (cherry-pick #12984)#13033
pvijayakrish merged 1 commit into
release/1.4.0from
gluo/cherrypick-12984-glm-image-nixl-stages-schema

Conversation

@GuanLuo

@GuanLuo GuanLuo commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Cherry-pick of fix(vllm): migrate GLM-Image NIXL config to the stages deploy schema #12984 to release/1.4.0.
  • GLM-Image AR→DiT disaggregation over NIXL dies on the first
    /v1/images/generations request: the AR stage's engine core exits with
    KeyError on the request id in gpu_model_runner._update_states, the request
    returns HTTP 500, and the deployment does not recover.
  • stage_configs/glm_image_nixl.yaml was the last config on vLLM-Omni's legacy
    stage_args schema, which applies no schema defaults. It pinned
    scheduler_cls: OmniARScheduler (the synchronous scheduler) while leaving
    async_scheduling unset, so the engine ran async scheduling against a sync
    scheduler. The sync scheduler does not populate
    CachedRequestData.all_token_ids, which the async resume path indexes
    unconditionally.
  • Migrating to the stages schema and dropping scheduler_cls lets vLLM-Omni
    resolve the matched scheduler / async_scheduling pair. Also retires the
    engine_input_source: [] workaround cherry-picked in fix(vllm): declare entry-stage engine_input_source in GLM-Image NIXL config (cherry-pick #12709) #12719, since input
    sources now come from the frozen pipeline.

Fixes DYN-3802

Original PR

Test plan

  • CI passes
  • Config resolves through vLLM-Omni's deploy loader to the intended matched
    pair: scheduler_cls=OmniARAsyncScheduler with async_scheduling=True
  • Manual 2-GPU single-node run via disagg_omni_glm_image_nixl.sh: both
    stages start, /v1/images/generations returns an image, and the AR engine
    core no longer dies on the first request

🤖 Generated with Claude Code


Open in Devin Review

…12984)

Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>
@GuanLuo
GuanLuo requested a review from a team as a code owner August 11, 2026 18:27
@github-actions github-actions Bot added fix backend::vllm Relates to the vllm backend labels Aug 11, 2026

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

Open in Devin Review

# defaults, and vLLM-Omni reads this field unconditionally.
engine_input_source: []
devices: ${oc.env:AR_GPUS,0}
tensor_parallel_size: ${oc.decode:${oc.env:AR_TP,1}}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 oc.decode on AR_TP relies on lazy interpolation when AR_TP is unset (DiT node)

tensor_parallel_size now goes through ${oc.decode:...}. OmegaConf's oc.decode raises TypeError when handed a non-string (the unquoted literal 1 in an interpolation default is parsed as an int, not a string), so this expression only survives if either the env var is set or the value is never resolved. On the DiT node, examples/backends/vllm/launch/disagg_omni_glm_image_nixl.sh does not export AR_TP, so stage 0's field would resolve to the int default if the loader eagerly resolves the whole file. The identical pattern already existed for num_gpus: ${oc.decode:${oc.env:DIT_TP,1}} on the AR node before this PR (and the author reports a successful 2-GPU run), which is the reason this is not flagged as a bug — but a cross-node run where AR_TP is unset on the DiT machine is worth an explicit check, and quoting the defaults (${oc.env:AR_TP,'1'}) would make it unconditionally safe.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

@datadog-official

This comment has been minimized.

@pvijayakrish
pvijayakrish merged commit 3534ffd into release/1.4.0 Aug 11, 2026
102 of 104 checks passed
@pvijayakrish
pvijayakrish deleted the gluo/cherrypick-12984-glm-image-nixl-stages-schema branch August 11, 2026 22:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend::vllm Relates to the vllm backend fix size/L

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants