fix(vllm): migrate GLM-Image NIXL config to the stages deploy schema (cherry-pick #12984) - #13033
Conversation
…12984) Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>
| # defaults, and vLLM-Omni reads this field unconditionally. | ||
| engine_input_source: [] | ||
| devices: ${oc.env:AR_GPUS,0} | ||
| tensor_parallel_size: ${oc.decode:${oc.env:AR_TP,1}} |
There was a problem hiding this comment.
🔍 oc.decode on AR_TP relies on lazy interpolation when AR_TP is unset (DiT node)
tensor_parallel_size now goes through ${oc.decode:...}. OmegaConf's oc.decode raises TypeError when handed a non-string (the unquoted literal 1 in an interpolation default is parsed as an int, not a string), so this expression only survives if either the env var is set or the value is never resolved. On the DiT node, examples/backends/vllm/launch/disagg_omni_glm_image_nixl.sh does not export AR_TP, so stage 0's field would resolve to the int default if the loader eagerly resolves the whole file. The identical pattern already existed for num_gpus: ${oc.decode:${oc.env:DIT_TP,1}} on the AR node before this PR (and the author reports a successful 2-GPU run), which is the reason this is not flagged as a bug — but a cross-node run where AR_TP is unset on the DiT machine is worth an explicit check, and quoting the defaults (${oc.env:AR_TP,'1'}) would make it unconditionally safe.
Was this helpful? React with 👍 or 👎 to provide feedback.
Summary
release/1.4.0./v1/images/generationsrequest: the AR stage's engine core exits withKeyErroron the request id ingpu_model_runner._update_states, the requestreturns HTTP 500, and the deployment does not recover.
stage_configs/glm_image_nixl.yamlwas the last config on vLLM-Omni's legacystage_argsschema, which applies no schema defaults. It pinnedscheduler_cls: OmniARScheduler(the synchronous scheduler) while leavingasync_schedulingunset, so the engine ran async scheduling against a syncscheduler. The sync scheduler does not populate
CachedRequestData.all_token_ids, which the async resume path indexesunconditionally.
stagesschema and droppingscheduler_clslets vLLM-Omniresolve the matched scheduler /
async_schedulingpair. Also retires theengine_input_source: []workaround cherry-picked in fix(vllm): declare entry-stage engine_input_source in GLM-Image NIXL config (cherry-pick #12709) #12719, since inputsources now come from the frozen pipeline.
Fixes DYN-3802
Original PR
byte-identical to the version merged to main.
Test plan
pair:
scheduler_cls=OmniARAsyncSchedulerwithasync_scheduling=Truedisagg_omni_glm_image_nixl.sh: bothstages start,
/v1/images/generationsreturns an image, and the AR enginecore no longer dies on the first request
🤖 Generated with Claude Code