[Core] RFC #4872 refactor stage_input_processors

[RyanYun09]

Purpose

This PR implements the stage_input_processors refactor tracked by RFC #4872. Before this change the module suffered from:

• No unified contract — two separate inspect.signature probe points were used to introspect processor callables, leaving the runtime contract implicit and fragile.
• Duplicated, semantically drifted helpers — equivalent utility logic was copy-pasted across modules with subtle behavioural differences.
• Scattered constants — magic strings and defaults were defined locally per module instead of a single source of truth.
• Empty-package surface — the public stage_input_processors package exported no stable API.
• Split async/sync placeholder paths — async-chunk and sync placeholder prompt building followed two divergent code paths and relied on hardcoded token ids.

The change is layered:

1. Contract layer (_dispatch.py): introduces OrchestratorInputContext plus four Protocols (placeholder prompt builder, diffusion builder, and two producer contracts), a single invoke_orchestrator_processor entry point, legacy signature adapters (C0–C4), and a DeprecationWarning (M0) for legacy positional contracts.
2. Registry (_registry.py): centralizes resolve_processor / validate_processor with loose/strict is_finished rules, suffix ↔ kind mapping, and a dead-processor three-state gate; the three existing resolution points (stage_init_utils, the worker mixin, and chunk_transfer_adapter) are unified through it.
3. Shared helpers (_common.py): consolidates 13 helper functions and deduplicates 10 modules by delegation, with a 77-case golden test suite locking the previously divergent semantics.
4. Centralized constants (_constants.py): moves scattered constants into a single module, backed by a consistency test.
5. Placeholder unification (P8a/P8b): unifies the placeholder prompt builders behind dual build_forward / build_prewarm entries; the prewarm path now resolves through the registered builder and hardcoded token ids are removed.
6. Tests: golden (77), registry (39), placeholder parity (10), and constants consistency (5).

Test Plan

vLLM Version: local CPU regression on 0.28.0 (arm64 wheel); repo CI (Dockerfile.ci / Buildkite) also runs vLLM 0.28.0

vLLM-Omni Commit: based on upstream main merge-base 232dbc82f (vLLM 0.28.0) + 31 commits: 15513fdbb, 68993b3bf, babc77dd4, d537e91bd, 5b628fbe8, d7377885d, 1460be497, d01562a83, b1a771801, 3eafd2733, 06fcb65ac, 2054347f3, ee0aac3ca, 48a4e7344, f44a02675, a690170bd, 7c323bcb9, 0a7387698, 2e2953a97, 249e6202f, 1003b26c8, 30b3cf087, 81ba7d6a5, ed5608313, 47a36dfb0, 073543019, a36dbdb65, 72852b558, ba23816c7, 490e438be, ad81ac3ac; PR branch HEAD = ad81ac3ac

31 commits span the original 25 (up to 29f17d4e) plus 4 rebase-damage repair commits + 2 CI hotfix commits + 2 latest review-round commits (490e438be regression tests for the prewarm override, ad81ac3ac the P1 fixes + AuK restore).

CI status at ad81ac3ac: DCO ✅ · build (3.11) ✅ · build (3.12) ✅ · pre-commit ✅ · docs/readthedocs ✅ (Buildkite pending)

Tests executed (vLLM 0.28.0, arm64 CPU container, PR HEAD ad81ac3ac):

• tests/model_executor/stage_input_processors/ (full directory) — 594 passed / 0 failed
• tests/config/test_config_factory.py — 330 passed / 0 failed
• tests/engine/test_orchestrator_stage_input_bridge.py — 12 passed / 0 failed
• tests/engine/ (CPU-marked) — 587 passed / 1 skipped / 2 deselected (GPU-only); this includes the 9 engine files that reference vLLM 0.28's CoreEngineLaunch (previously uncollectable on 0.27.1) — 190 CPU cases passed
• tests/diffusion/ files previously blocked on vLLM 0.28 (stage_diffusion_proc-dependent) — 80 passed, 1 pre-existing failure in test_diffusion_streaming_output (asyncio teardown race, not introduced by this PR)
• tests/model_executor/stage_input_processors/test_registry.py / test_placeholder_parity.py / test_constants_consistency.py — included in the 594 above
• Startup-level pipeline validation — 36 model imports / 80 processor-path resolutions passed

The 4 rebase-damage repair commits + 2 CI hotfix commits (47a36dfb0, 073543019, a36dbdb65, 72852b558, ba23816c7) are scoped to engine/merge code and do not touch any stage_input_processors logic or test files. The latest review-round commit ad81ac3ac touches stage_input_processors/auk.py (AuK text-cond restore, see below) and lifts the SIP full suite to 594 passed / 0 failed under the same vLLM 0.28.0 environment.

Test Result

CPU container (linux/arm64, vLLM 0.28.0): all PR-relevant suites above ran with 0 failures. GPU/NCCL- and memory-sensitive cases (pi0, distributed comms, norm layers) are environment-limited locally and are validated by CI (Buildkite, vLLM 0.28.0).

Latest Review Round — @amy-why-3459's two P1s + AuK restore (commit ad81ac3ac)

Commit ad81ac3ac resolves the two P1 compatibility regressions from @amy-why-3459's latest review (CHANGES_REQUESTED, review 5255823592, 2026-09-19) and restores the AuK text-cond semantics. 6 files changed, +211/−8; DCO-signed (Signed-off-by: RyanYun09).

P1 #1 — signature-cache initialization restored (chunk_transfer_adapter.py)

OmniChunkTransferAdapter.__init__ re-initializes self._processor_accepts_step_tokens = {} before super().__init__(model_config), so _accepts_new_token_ids() (invoked on every custom async-chunk producer send) no longer raises AttributeError. Both a normal chunk payload and the final/segment-finished flush reach the connector; a production-path regression test was added through _send_single_request() (tests/distributed/omni_connectors/test_chunk_transfer_adapter.py).

P1 #2 — target-model configuration injection preserved (_dispatch.py / stage_engine_core_client.py)

• OrchestratorInputContext gains target_model_config and next_stage_hf_config;
• StageEngineCoreClient.process_engine_inputs fills them (getattr-guarded);
• the legacy call-shape adapters inject them into processors that declare the matching keyword-only parameters; sampling_params is excluded from the C3 shape to avoid a duplicate-kwarg conflict.

JoyAI joyai_action_to_tts() now binds its keyword-only target_model_config, and Nemotron's thinker2talker_token_only() receives next_stage_hf_config as its fourth argument instead of being misread as streaming_context — the native talker placeholder keeps its configured talker_init_len (e.g. 37). Regression tests added through process_engine_inputs() (tests/distributed/omni_connectors/test_chunk_transfer_adapter.py, tests/model_executor/stage_input_processors/test_nemotron_voicechat.py).

AuK text-cond restore (auk.py)

_to_cpu_tensor regains its AuK-wide semantics (recursive list → torch.cat(dim=0), numpy / array-like → torch.as_tensor, empty list → ValueError), fixing the accidental narrowing to _common.to_cpu_tensor introduced by commit 073543019; the three TestTextCondExtraction tests pass again.

Updated test evidence

• tests/model_executor/stage_input_processors/ (full directory) — 594 passed / 0 failed
• Other PR-relevant suites (config_factory 330, bridge 12/12, engine CPU-marked, diffusion) unchanged and green; GPU/NCCL- and memory-sensitive cases remain CI-validated (Buildkite, vLLM 0.28.0).

NPU E2E validation — refreshed on PR head 29f17d4e with vLLM 0.28.0 (Ascend A3) ✅

Supersedes the earlier a0ecfe73 refresh. The 6 review comments from @hsliuustc0106 are addressed in 6 commits (d3c24da5 c7d8a9ac 520deb61 35eb88ea 2e12e139 29f17d4e); re-validated on the new head.

Item	Value
Image	vllm-ascend:v0.28.0-a3 (CANN 9.1.0, vLLM 0.28.0 source-editable, vllm-ascend 0.20.2rc line, triton-ascend 3.2.2, torch 2.10.0+cpu / torch_npu 2.10.0.post4)
Hardware	Ascend A3 ×2 (4 chips, 64GB HBM/chip), NPU driver 26.0.rc1
vllm-omni	branch rfc4872-stage-input-processors-refactor @ 29f17d4ea705
Model	Qwen3-Omni-30B-A3B-Instruct, 3-stage qwen3_omni_moe.yaml (thinker TP2 / talker / code2wav), async_chunk=true, PIECEWISE cudagraph

Result: 11/11 functional checks PASS — no functional regression vs the a0ecfe73 round and the v0.27.1 baseline.

• Pipeline: all 3 stages loaded, weight load 15/15 shards, ready in 272s, HBM ~3×40GB + 2.9GB; log 0 error / 0 traceback / 0 "Event loop is closed"
• Review comments, NPU-side evidence: DeprecationWarning 0 · dead-processor false positives 0 · Signature.bind errors 0 · prewarm fallback warnings 0; recursive per-item ensure_list path not exercised by this Qwen3-Omni pipeline (N/A on NPU, golden-locked by CPU tests)
• Shim files untouched by the new commits (git diff a0ecfe73..29f17d4e -- vllm_omni/platforms/npu/worker/ is empty)

Shim attribution (not part of this PR): on this same branch, removing the 3 getattr shims reproduces the exact gap (AttributeError: 'NPUARModelRunner' object has no attribute 'calculate_kv_scales' at npu_ar_model_runner.py:718 → EngineDeadError → /health 503); re-applying them restores a fully passing service. The gap pre-exists on main's shared code: vLLM 0.28.0 base classes lack both attributes, the PR diff contains 0 touching changes to those lines, and the bare reference at :718 is byte-identical on main 3d035bfa (merge-base). (main HEAD itself dies earlier at import time under this image; the PR branch carries its own vLLM 0.28.0 import-compat layer — compatibility better than main's.)

BEFORE SUBMITTING: read CONTRIBUTING.md and run the precheck-pr skill with the code agent for a self-check against project conventions.
(anything written below this line will be removed by GitHub Actions)

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

[vllm-omni-review-bot]

This PR appears to belong to: docs/design/module/input_output_modality_contracts.md.

Module owners: @Sy0307 @amy-why-3459 @Gaohan123 @alex-jw-brooks

@RyanYun09, please review your own changes and leave a short self-review comment describing what you checked. PRs without author self-review may not be assigned a reviewer.

Please take a look when you have a chance. If you would like an automated review, mention @vllm-omni-review-bot in a comment.

[amy-why-3459]

The layering is right: one OrchestratorInputContext, one resolve_processor behind the three importlib sites, golden-then-delegate helpers, dual-entry P8b on *_token_only, and P8c left alone. Producer kwargs stayed load-bearing. Request changes on three runtime diffs, not on the docs.

1. diffusion/output_formatter._ensure_list is not a stage-input helper. It used to wrap a non-list primary payload as [x]. _common.ensure_list flattens tensors, iterates dict keys, and will walk a PIL Image by rows. format_diffusion_outputs then passes that list through as images=outputs. RFC P3 only allows a _common switch after that call site's own semantics are golden-locked. Put the wrap back, or add a wrap-only helper. Do not reuse the processor flatten.

2. extract_legacy_stage_metadata swallows every resolve_processor failure and continues with custom_process_input_func=None. The stage then hits _default_process_engine_inputs (downstream prompt = upstream token_ids). A bad path used to fail at import. RFC P2 and the adding-omni text are fail-fast. M0 is a DeprecationWarning on C2/C3/C4, not a silent drop of the configured hook. Let ProcessorValidationError / ImportError / AttributeError raise. The mixin candidate chain may still skip a mismatched _full_payload probe.

3. P8b happy-path prewarm is len(stage0_prompt); the inline fallback and adapter.compute_talker_prompt_ids_length still run the Qwen chat-template scan. RFC 8a is one scan, two input shapes (full = prompt+generated, stage0_only = stage-0 ids only). The golden prompt is 15 from the scan and 10 from len(). test_forward_prewarm_golden_relationship locks that split. A chat template with an assistant marker under-reserves until connector fixup; a long system prompt over-reserves; the same request is 10 or 15 depending on whether resolve succeeded. Make stage0_only the scan on the stage-0 list (what the adapter already does) and make the builder and the fallback return the same number. The single-user-segment case can stay 6==6. You can still del downstream_stage_id.

The A3 table is 0.1.dev2687+g350c86591, not e38b0d1c (that SHA is pre-ef2e088 merge). CPU suites are on vLLM 0.27.1; CI is 0.28.0. Do not treat that run as this head.

Wrap once at resolve — invoke_orchestrator_processor re-wraps every request, so the C2 thinker2talker_token_only shell emits a DeprecationWarning per forward. Delete the leftover _compute_talker_prompt_ids_length. Move _assert_codec_token_ids_consistent off import. Wire dead_processor_hint at startup. The adding-omni snippet shows C1 thinker2talker_token_only; the function is still the four-arg shell. Author field is vllm-omni RFC 4872 <rfc4872@local> — rebase with a real identity. Self-review is still missing: say what you checked on the formatter wrap, resolve fail-fast, and the prewarm length.

[vllm-omni-review-bot]

Omni ReviewBot triage note

Automated triage of commit 7a77d8318b79 produced:

• Priority: high. Prompt maintainer attention is suggested.

These are automated triage suggestions only — the final decision belongs to the maintainers.

[amy-why-3459]

Reviewed head 7a77d8318b793862e1bc0653e28501893ccf41fe. Requesting changes for the Voxtral codec-frame regression described inline.

Targeted validation: 1,130 tests passed, plus 3 subtests, covering stage input processors, the orchestrator stage-input bridge, chunk transfer adapter, connector mixin, and stage metadata. A separate CPU base/head reproduction executes the production compute_mm_logits() method with a simulated acoustic transformer and passes its output through the real worker payload builder: base produces an audio payload; head raises ValueError: unexpected audio_codes shape: (1, 1, 8). This is a function/payload reproduction, not model-weight or end-to-end inference validation.

Separate validation gap: please provide complete repository unit-test evidence tied to this SHA (and the updated head after fixing the regression), including the CPU UT shards defined in .buildkite/cuda/test-ready.yml and applicable required hardware/backend-specific UT jobs. Attach exact commands, environment/dependency versions, passed/failed/skipped/deselected/collection-error counts, and CI or complete-log links. Explain skips and unavailable jobs; if failures are pre-existing, include a same-environment base/head comparison. Existing complete CI evidence on the same revision is sufficient; targeted green tests alone do not establish full-suite coverage. Merge readiness remains unestablished until this evidence is supplied or maintainers explicitly disposition the coverage gap.

[amy-why-3459]

[P1] Preserve Voxtral's three-dimensional codec frame shape

VoxtralTTSAudioGeneration.compute_mm_logits() constructs audio_list by splitting audio_codes.unsqueeze(1) along the batch dimension, so each per-request codec tensor has shape [1, 1, C]. The worker payload builder preserves this shape. The previous _extract_last_frame() flattened it successfully, but the new extract_last_codec_frame() accepts only 1-D and 2-D tensors and raises on this normal model output.

I reproduced this with the production model method (using a simulated acoustic transformer) and the actual build_mm_cpu() / build_omni_mm_payload() path: the per-request shape is (1, 1, 8), base emits the expected audio payload, and this head raises ValueError: unexpected audio_codes shape: (1, 1, 8) before buffering any frame. The transfer adapter catches the exception and drops the audio payload; completion can therefore forward only an empty terminal marker.

Please preserve Voxtral's frame-flattening semantics, or normalize this shape in its model-specific adapter without changing other models' last-row semantics. Add regression coverage using [1, 1, C] through payload construction, chunk emission, and final flush; the existing 1-D mock inputs do not cover the actual model output.

[vllm-omni-review-bot]

Omni ReviewBot attempt record

Review attempt ended as failed (failed; retrying strict/cursor/cursor-grok-4.6-high in 120s (try 2 of 3)).

[vllm-omni-review-bot]

Omni ReviewBot attempt record

Review attempt ended as failed (failed; retrying strict/cursor/cursor-grok-4.6-high in 600s (try 3 of 3)).

[vllm-omni-review-bot]

Omni ReviewBot attempt record

Review attempt ended as failed (failed; falling back to direct/cursor/auto).