[Config Refactor][2/N] Pipeline + Deploy Config Schema

[lishunyang12]

RFC: #2072

Motivation

Before this refactor, vLLM-Omni's multi-stage pipelines mixed topology (which stages exist, how they wire, what functions they call) and deployment parameters (TP size, memory budgets, device placement, connectors) into a single stage_configs/<model>.yaml per platform. Adding a new platform meant editing N files; changing a max_num_seqs meant forking the whole YAML; model developers and deployment engineers edited the same file with different concerns in mind.

This PR implements RFC #2072 — splitting the legacy YAML into two layers connected by a runtime merge with documented precedence.

Design

Before vs. after

flowchart LR
    subgraph Before["Before — single YAML per platform"]
        direction TB
        Y1["stage_configs/qwen3_omni_moe.yaml<br/>(topology + params + connectors)"]
        Y2["platforms/npu/stage_configs/qwen3_omni_moe.yaml<br/>(full copy for NPU)"]
        Y3["platforms/rocm/stage_configs/qwen3_omni_moe.yaml<br/>(full copy for ROCm)"]
    end
    subgraph After["After — split by concern"]
        direction TB
        P["models/qwen3_omni/pipeline.py<br/>(frozen topology — developer-owned)"]
        D["deploy/qwen3_omni_moe.yaml<br/>(CUDA defaults — deployer-owned)"]
        DN["deploy/qwen3_omni_moe.yaml<br/>:platforms.npu.stages<br/>(platform deltas inline)"]
    end
    Before --> After

Loading

• Pipeline (models/<name>/pipeline.py) calls register_pipeline(PipelineConfig(...)) at import time. Frozen — deploy cannot reshape the graph.
• Deploy (deploy/<model>.yaml) carries per-stage TP size, GPU memory, device placement, connectors, and platform deltas.
• A single CUDA default yaml with platforms: { npu, rocm, xpu } sections replaces three parallel files.

Two-level config objects

Aspect	PipelineConfig	DeployConfig
Mutability	frozen=True	mutable (platform/CLI override)
Source	Python (pipeline.py)	YAML (deploy/*.yaml)
Owner	model developers	deployment engineers
Contains	stages, edges, processor funcs, sampling constraints	TP/mem/devices, connectors, platform deltas

Precedence chain

flowchart LR
    D1["Parser defaults"] -->|weakest| D2["Base deploy YAML"]
    D2 --> D3["Overlay YAML<br/>via base_config:"]
    D3 --> D4["Platform section<br/>platforms.npu.stages"]
    D4 --> D5["Global CLI<br/>--gpu-memory-utilization"]
    D5 -->|strongest| D6["Per-stage CLI<br/>--stage-overrides JSON"]

Loading

User-typed keys are tracked via _cli_explicit_keys (parser-aware: walks parser._actions so --disable-X → dest=enable_X and alias flags resolve correctly) so argparse defaults do not silently overwrite YAML values.

CLI flag routing — OrchestratorArgs

make_arg_parser flattens uvicorn / FastAPI / engine / orchestrator flags into a single namespace. The old code maintained two hardcoded frozensets (~49 strings total) as denylists — fragile. OrchestratorArgs replaces them with a dataclass; split_kwargs classifies each flag by dataclass membership; CI invariants in tests/test_arg_utils.py catch unclassified flags at test time.

Auto-discovery

No more hardcoded PIPELINE_MODELS / _ARCHITECTURE_MODELS dicts. _discover_all_pipelines scans model_executor/models/*/pipeline.py and registers them; a contributor adding a new model just drops a pipeline.py.

Summary of Changes

Area	Change
New modules	vllm_omni/engine/arg_utils.py (OrchestratorArgs + SHARED_FIELDS + split_kwargs), vllm_omni/deploy/*.yaml (3 default deploy configs + CI overlays)
New pipelines	models/qwen2_5_omni/pipeline.py, qwen3_omni/pipeline.py, qwen3_tts/pipeline.py
New CLI flags	--deploy-config, --stage-overrides, --async-chunk / --no-async-chunk
Removed hardcoded state	INTERNAL_STAGE_OVERRIDE_KEYS, SERVER_ONLY_KEYS, PIPELINE_MODELS, _ARCHITECTURE_MODELS — replaced by dataclass-derived invariants and auto-discovery
Refactorings	merge_pipeline_deploy split into 4 single-responsibility helpers (SLAP); _apply_platform_overrides deduplicated; execution_type → (stage_type, worker_type) lookup table; parser-aware detect_explicit_cli_keys
Schema clean-up	8 pipeline-wide engine settings (trust_remote_code, distributed_executor_backend, dtype, quantization, enable_prefix_caching, enable_chunked_prefill, data_parallel_size, pipeline_parallel_size) moved from per-stage to top-level DeployConfig
Validation	merge_pipeline_deploy raises if async_chunk=True but no stage declares an async handler; get_scheduler_cls raises on invalid stage_id / unmapped execution_type; _deep_merge_stage warns on type-mismatch clobber; --stage-configs-path and --deploy-config are mutex; scheduler map stores class refs (rename fails at import)
Legacy paths retained	--stage-configs-path (deprecated in help text, to be removed in 2c); ModelPipeline / StageConfig / _parse_pipeline_yaml preserved for not-yet-migrated models
Tests	tests/test_arg_utils.py (15 invariants incl. BVA), expanded tests/test_config_factory.py (+644 lines)
YAML style	Deploy YAMLs carry only non-default values; every field falls back to the StageDeployConfig dataclass default (single source of truth at vllm_omni/config/stage_config.py)
CLI entry	Omni.from_cli_args(args, parser=parser) / AsyncOmni.from_cli_args(args, parser=parser) mirror OmniEngineArgs.from_cli_args; optional parser= enables accurate _cli_explicit_keys resolution
Docs	docs/configuration/stage_configs.md rewritten with unified schema tables, connector schema, worked override precedence example; examples/online_serving/qwen3_tts/README.md gains a Sync vs async-chunk mode section

Test Plan

1. Unit tests + smoke scripts (CPU-only)

pytest tests/test_config_factory.py -v
pytest tests/test_arg_utils.py -v
python tools/smoke_config_loading.py        # 19 checks
python tools/smoke_cli_explicit_keys.py     # 13 checks

2. E2E launch matrices (GPU box)

qwen2_5_omni

BASE="vllm serve Qwen/Qwen2.5-Omni-7B --omni --port 8091"

# Default — auto-loads vllm_omni/deploy/qwen2_5_omni.yaml
$BASE

# Global CLI flag (applies to every stage; explicit beats yaml)
$BASE --max-model-len 16384

# Per-stage override via JSON
$BASE --stage-overrides '{"1":{"gpu_memory_utilization":0.5},"2":{"max_num_batched_tokens":16384}}'

# Explicit precedence: per-stage beats global
$BASE --max-num-seqs 4 --stage-overrides '{"1":{"max_num_seqs":8}}'

# BooleanOptionalAction flags
$BASE --enable-prefix-caching
$BASE --no-enable-prefix-caching
$BASE --no-async-chunk

qwen3_omni_moe

BASE="vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091"

# Default — auto-loads vllm_omni/deploy/qwen3_omni_moe.yaml
$BASE
$BASE --stage-overrides '{"1":{"gpu_memory_utilization":0.5}}'

# Multi-node overlay (README has a worked example)
$BASE --deploy-config /path/to/my_qwen3_omni_multinode.yaml

qwen3_tts (async vs sync codec, both from one yaml)

BASE_TTS="vllm serve Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice --omni --port 8091"

# Async chunk ON (yaml default) — chunked streaming
$BASE_TTS

# Async chunk OFF — same pipeline, alternate processor function selected
# automatically by merge_pipeline_deploy based on deploy.async_chunk.
$BASE_TTS --no-async-chunk

# Batched throughput via per-stage overrides (replaces the deleted qwen3_tts_batch.yaml)
$BASE_TTS --stage-overrides '{"0":{"max_num_seqs":4,"gpu_memory_utilization":0.2},"1":{"max_num_seqs":4,"gpu_memory_utilization":0.2}}'

3. Server flag isolation (regression check for #873-class bugs)

vllm serve Qwen/Qwen2.5-Omni-7B --omni --port 8091 \
    --host 0.0.0.0 \
    --served-model-name my-omni \
    --api-key secret123 \
    --allowed-local-media-path /tmp/

Expected: clean startup. A TypeError: unexpected keyword argument 'host' from OmniEngineArgs.__init__ would indicate server flags leaking into the per-stage engine path.

Review feedback addressed

19 threads from @alex-jw-brooks resolved — correctness fixes (mutex validation for --stage-configs-path / --deploy-config, async_chunk handler check, get_scheduler_cls error paths, deep-merge clobber warning, parser-aware flag detection), cleanups (removed redundant qwen3_tts_no_async_chunk alias, dead get_stage_config wrappers in 4 test files), and doc clarifications (logical device IDs, engine_extras rationale). See #2887 for deferred follow-ups (hardware auto-sizing, model-instance-driven config values, override type validation, central pipeline registry).

What ships in follow-ups

• 2c: remove --stage-configs-path and legacy ModelPipeline / _parse_pipeline_yaml; migrate remaining legacy models (fish_speech, cosyvoice3, mimo_audio, voxtral_tts) to the registry; split stage_config.py (~1200 LOC) into focused modules once the legacy surface is gone.
• Tooling PR: carve tools/smoke_*.py into its own PR.
• [Follow-up] Deploy/pipeline config follow-ups from #2383 #2887 tracks the other review follow-ups.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 929007a841

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Whitelist stage override keys before copying CLI kwargs

This loop forwards every non-None CLI kwarg that is not in a small denylist, so non-engine server flags can leak into per-stage engine_args. In the OpenAI server path, AsyncOmni is built from vars(args), which includes API/uvicorn options; once forwarded here, they eventually hit AsyncOmniEngineArgs(model, **engine_args) and can fail with unexpected-keyword errors for migrated models. Please filter by an allowlist of engine/runtime keys (or a parser-backed schema) instead of forwarding arbitrary kwargs.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Preserve CLI async_chunk override in resolved config

CLI overrides can set async_chunk in engine_args, but this assignment overwrites that value with self.async_chunk, which is currently sourced from deploy YAML (deploy.async_chunk) in _resolve. As a result, --async-chunk cannot actually override deploy defaults, breaking the documented precedence and preventing users from toggling async mode at runtime.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Forward tokenizer override for registered-model path

Marking tokenizer as an internal key causes it to be dropped from all stage overrides in the new factory path. The legacy YAML path explicitly injected tokenizer into stage engine args, so this is a behavior regression: users passing --tokenizer for migrated models (like qwen3_omni_moe) will silently run with the default tokenizer instead of the requested one.

Useful? React with 👍 / 👎.

[lishunyang12]

@hsliuustc0106 @david6666666 @wuhang2014 My P0 priority will be on this pr this week.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1e1c756f8b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Preserve explicit global CLI overrides in registry path

Filtering cli_overrides down to only stage_<id>_* keys means explicit global engine flags are silently ignored for migrated models (for example --max-model-len, --max-num-seqs, or --gpu-memory-utilization). As a result, users can no longer tune all stages via normal CLI arguments and must rewrite everything into per-stage JSON overrides, which is a behavior regression versus the legacy path.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Parse deploy connector schema before using deploy path as config

This new branch forwards deploy YAML paths as config_path, but stage initialization still loads transfer connectors through load_omni_transfer_config_for_model, whose parser expects legacy runtime.connectors + stage_args structure. New deploy files use connectors + stages, so connector specs/extras are dropped; in async-chunk or distributed setups, custom connector backends and connector tuning in deploy configs will not take effect.

Useful? React with 👍 / 👎.