[Core][Frontend] Unified Full-duplex Framework

[chickeyton]

PR1 of RFC #7181

Purpose

Changes

1. Make duplex serving layer a thin layer, mirgrate the duplex logics like DuplexSession, and session runner into engine layer
   Seperate the turn based API and engine logics and the duplex API and engine logics by adding DuplexOmni, DuplexOmniEngine and DuplexOrchestrator. Mirgate the duplex API and logics from AsyncOmni,AsyncOmniEngine and Orchestrator to the newly added classes.
2. Make /v1/duplex an alias of /v1/realtime?duplex=1 and drop the native_duplex parameter in new session message. A duplex model keeps /v1/chat/completions: DuplexOmni extends AsyncOmni, so the ordinary chat service runs on the same engine and the same weights, gated on the model declaring DuplexCapabilities.supports_chat_completions. Only /v1/realtime?duplex=1 (/v1/duplex), /v1/chat/completions, /v1/models and /health are supported by this Full-duplex framework; every other turn-based route reports "not available". This amends RFC decision D9 ("duplex-only API server").
3. Add InlineDuplexClient, same usage as DuplexClient but relay on DuplexOmni object.
4. Drop the concept of incarnation, make the session id the new session be always allocated by server, simplify the session creation and session resume logics.
5. Fit MiniCPM-o 4.5 into the framework

Temporary Changes

1. Since Qwen3-Omni, PersonalPlex and Nemotron VoiceChat are to be fitted into the Unified Full-duplex Framework in PR5, PR2 and PR4 in the begining RFC, their tests are skipped by adding:

test/model_executor/models/personaplex/duplex/conftest.py
test/model_executor/models/nemotron_voicechat/duplex/conftest.py

the above files are going to be removed after PR2 an PR4

and

test/entrypoints/openai_api/test_qwen3_omni_realtime_websocket.py

Other bugfixes

Three defects this benchmark found, all fixed here

1. The terminal-event guard rejected our own close.
2. Stage-0 engine metrics were dropped, so TPOT could not be computed.
3. 1q1a_math deadlocked, and the model appeared to generate 10x too much audio.

Benefits

1. A duplex session now has exactly one owner, owned by engine layer, more compact and easly to read code.

2. The turn-base (e.g. HunyuanImage model) modules no longer carries duplex code, AsyncOmni-AsyncOmniEngine-Orchestrator stack become less bulky.

3. Simplified usage and documenations by dropping all the unnecessary fallback modes, leaving the two surfaces a duplex session actually backs: v1/realtime?duplex=1 and v1/chat/completions.

4. Advantages for AURA's intergration (may also true to other duplex models):

• One in-process session object that can own AURA's history, instead of
  today's registry that is written in the API server process and read again
  in the Stage 1 worker process.
• Interrupt, cancel, barge-in, resume and multi-session with less to wire
  and debug. The baseline duplex framework implements the same behaviours
  and also binds forwarded stage requests to the session, so AURA would
  inherit the same feature list on either tree once it is ported off the
  streaming-video handler (which has none of them). The difference is where
  they live: one handler in the runner reaching all four stages, versus the
  serving mixins, the RPC boundary and the engine control plane with fence
  re-validation on both sides.
• A single, validated plugin seam (the one DuplexModelPlugin the engine
  loads and hands to the orchestrator) where AURA's turn, silent and tool
  rules live in AURA's own file, instead of the baseline's engine extension
  plus serving adapter pair.

New Architecture

              Python users                     WebSocket clients              HTTP clients
        ┌──────────────────────┐        ┌────────────────────────┐    ┌──────────────────────┐
        │  InlineDuplexClient  │        │      DuplexClient      │    │   OpenAI chat SDK    │
        │  (DuplexClientBase)  │        │   (DuplexClientBase)   │    │ /v1/chat/completions │
        └──────────┬───────────┘        └───────────┬────────────┘    └──────────┬───────────┘
                   │   DuplexSessionHandle          │ /v1/realtime?duplex=1 (JSON)│
                   │                                ▼                             │
                   │              ┌──────────────────────────────────────┐   thin layer
                   │              │ OmniDuplexSessionHandler             │        │
                   │              │  websocket I/O, DuplexCommand.from_  │        ▼
                   │              │  realtime(), event.to_realtime(),    │  OmniOpenAIServingChat
                   │              │  attachment / resume tokens / replay │  (unchanged; gated on
                   │              └──────────────────┬───────────────────┘  supports_chat_completions)
                   ▼                                 ▼   DuplexSessionHandle        │
        ┌──────────────────────────────────────────────────────────────────────────────┐
        │ DuplexOmni(AsyncOmni)                      [entrypoints/duplex_omni.py]       │  thin layer
        │   open_session / get_session / resume_session / close_session                │
        │   DuplexSessionHandle: submit(DuplexCommand) / events()                      │
        │   inherited from AsyncOmni: generate(), EngineClient — the chat service runs  │
        │   on this same object, one engine and one set of weights                     │
        └──────────────────────────────────────┬───────────────────────────────────────┘
                                               │ engine.open/close/resume/touch_session (RPC), submit_command (one-way)
                                               ▼
        ┌──────────────────────────────────────────────────────────────────────────────┐   one single Duplex Session owner
        │ DuplexOmniEngine(AsyncOmniEngine) [engine/duplex_omni_engine.py, thin]        │
        │   message construction + queue put/RPC; creates DuplexOrchestrator            │
        │   (OmniEngineBase: stage init, orchestrator thread, queues, RPC router)       │
        │  └─ DuplexOrchestrator(Orchestrator)            [engine/duplex_orchestrator.py]│
        │       stage forwarding/prewarm/cleanup with session-owned policy              │
        │       _dispatch_message: session messages here, everything else falls through │
        │                          to Orchestrator, so one engine also admits requests  │
        │       ├─ DuplexSessionManager      admission, lease reaper, open/close/resume, │
        │       │                            dispatch of commands to runners            │
        │       └─ DuplexSessionRunner ×N    ONE session state (DuplexEngineSession),   │
        │                                    ordered mailbox, append planning + stage   │
        │                                    submit, data-plane decisions, response /   │
        │                                    playback / history, VAD, typed events      │
        │          [engine/duplex/session/]  runner + manager + the collaborators it     │
        │                                    delegates to: SessionControl, SessionEmitter│
        │                                    ModelChannel, AppendAttempt, helpers        │
        │       plugin: DuplexModelPlugin    (pipeline.duplex_plugin, one class/model)  │
        └──────────────────────────────────────────────────────────────────────────────┘

Class relationships:

OmniBase                        OmniEngineBase                       OrchestratorBase
 ├─ Omni        (sync, turn)     └─ AsyncOmniEngine  (turn requests)  └─ Orchestrator        (turn-based admission)
 └─ AsyncOmniBase                    └─ DuplexOmniEngine (+session)       └─ DuplexOrchestrator  (hosts DuplexSessionManager)
     └─ AsyncOmni   (EngineClient)  _create_engine -> AsyncOmniEngine   _create_orchestrator -> Orchestrator
         └─ DuplexOmni              _create_engine -> DuplexOmniEngine  _create_orchestrator -> DuplexOrchestrator

The duplex classes extend their turn-based counterparts rather than sitting
beside them, so one engine serves a duplex session and an ordinary request at the
same time. The dependency runs duplex -> turn-based and never the reverse, so
Orchestrator, AsyncOmniEngine and AsyncOmni still carry no duplex code and
tests/engine/test_duplex_import_boundary.py still holds. Two tests pin the
relationship itself (test_the_duplex_stack_extends_the_turn_based_one,
test_an_unrecognised_message_reaches_the_turn_based_handler), because a later
tidy-up back into siblings would silently stop a duplex server serving chat.

Verified on hardware rather than argued: a websocket duplex session held open and
mid-turn from 0.07 s to 4.31 s, with two /v1/chat/completions answering inside
that window, on one process and one set of weights.

Test Plan

vLLM Version: 0.29.0 · Hardware: 1× NVIDIA L20X (143 GB) per run, test server 1 · Venvs: .venv-dr2-v029 (CPU sweeps), .venv-fd3-v029 (GPU serving, torch 2.13.0+cu130, cloned from the 0.28.0 GPU venv with vLLM upgraded — the branch needs 0.29.0 since the #7230 rebase)

1. Unittests — refactored duplex framework

pytest -q -m "not gpu" \
  tests/engine/duplex/ \
  tests/entrypoints/duplex/ \
  tests/entrypoints/openai_api/test_duplex_api_server.py \
  tests/entrypoints/openai/test_duplex_session_attachment.py \
  tests/entrypoints/test_duplex_omni.py \
  tests/entrypoints/test_omni_entrypoints.py \
  tests/entrypoints/test_omni_base_profiler.py \
  tests/engine/test_duplex_orchestrator.py \
  tests/engine/test_duplex_omni_engine.py \
  tests/engine/test_duplex_import_boundary.py \
  tests/clients/ \
  tests/worker/test_native_duplex_hooks.py \
  tests/benchmarks/patch/test_patch.py \
  tests/benchmarks/test_omniinteract.py

1b. Everything else under tests/engine, tests/entrypoints, tests/clients

The lists above enumerate paths, so a test this refactor moved out from under
stays unrun. This sweep is what catches that, and it is compared against the
same run on main rather than against zero — the tree has pre-existing
failures that are not this PR's to fix.

pytest -q -m "not gpu and not cuda" tests/engine/ tests/entrypoints/ tests/clients/
# and the same command in a checkout of main, for the baseline

2. Unittests — MiniCPM-o 4.5

pytest -q -m "not gpu and not cuda" \
  tests/model_executor/models/minicpmo_4_5/ \
  tests/e2e/online_serving/test_minicpmo_realtime_duplex_drivers.py

3. MiniCPM-o 4.5 E2E and barge-in test

E2E suite — two CI tiers, each starting its own three-stage single-GPU server:

pytest -q tests/e2e/online_serving/test_minicpmo_4_5_duplex.py \
  -m "core_model and cuda"     --run-level core_model
pytest -q tests/e2e/online_serving/test_minicpmo_4_5_duplex.py \
  -m "advanced_model and cuda" --run-level advanced_model

Barge-in — the model must stop mid-sentence when the user talks over it:

sequenceDiagram
    autonumber
    participant U as user (client script)
    participant A as assistant (server + model)

    Note over U,A: GREET
    U->>A: start a voice session (with the reference voice)
    A-->>U: ready

    Note over U,A: ASK
    U->>A: ask a question (speak the question WAV)
    Note right of A: decides on its own to answer<br/>(no VAD — the model chooses)
    A-->>U: starts answering aloud

    Note over U,A: INTERRUPT
    Note over U: listens for ~2 seconds
    par both talk at once
        U->>A: interrupts with a follow-up
    and
        A-->>U: still speaking the first answer
    end

    alt the interruption is substantial
        Note right of A: stops mid-sentence (barge-in),<br/>first answer is cut off
    else it was just a short remark
        Note right of A: finishes the first answer,<br/>then takes the follow-up
    end

    Note over U,A: ANSWER AGAIN
    A-->>U: answers the follow-up aloud

    Note over U,A: HANG UP
    U->>A: goodbye (close the session)
    Note over U: saves one recording per answer<br/>plus a summary of what happened

Loading

start server

# defaults: MODEL=openbmb/MiniCPM-o-4_5, PORT=8099 — override via env
PORT=8098 examples/online_serving/minicpmo/barge_in_serve.sh
# which execs:
#   vllm-omni serve openbmb/MiniCPM-o-4_5 --omni \
#       --deploy-config vllm_omni/deploy/minicpmo_4_5.yaml \
#       --trust-remote-code --host 0.0.0.0 --port 8098

client

python examples/online_serving/barge_in_client.py \
    --url ws://127.0.0.1:8098/v1/realtime \
    --model openbmb/MiniCPM-o-4_5 \
    --ref-audio "$HF_HOME/hub/models--openbmb--MiniCPM-o-4_5/snapshots/<rev>/assets/HT_ref_audio.wav" \
    --question-wav tests/assets/minicpmo_4_5/response_required_16k.wav \
    --interrupt-wav tests/assets/minicpmo_4_5/soft_interrupt_16k.wav \
    --output-dir ./barge_in_out

This PR amends RFC decision D9: a duplex model now serves /v1/chat/completions
as well, on the same engine, so the same server is probed for the route
alongside the barge-in run:

curl -s -o /dev/null -w "%{http_code}\n" -X POST http://127.0.0.1:8098/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"openbmb/MiniCPM-o-4_5","messages":[{"role":"user","content":"hi"}]}'

4. E2E benchmark with MiniCPM-o 4.5

OmniInteract realtime benchmark, one deterministic case per subset,
--num-prompts 1 --max-concurrency 1 --num-warmups 0 — the setup of PR #6522's
"E2E Benchmark Result" section. Run twice: once on this branch and once on its
merge base main @ 767cc797, same box and venv, so the two tables are a
like-for-like comparison rather than a quote from an older run.

vllm-omni bench serve --omni --trust-remote-code \
  --backend openai-realtime-duplex --dataset-name omniinteract \
  --dataset-path "$OMNIINTERACT_ROOT" --model openbmb/MiniCPM-o-4_5 \
  --base-url http://127.0.0.1:8133 --endpoint /v1/realtime \
  --omniinteract-subsets "$SUBSET" \
  --omniinteract-ref-audio ".../MiniCPM-o-4_5/assets/HT_ref_audio.wav" \
  --omniinteract-output-dir ./artifacts --num-prompts 1 \
  --max-concurrency 1 --num-warmups 0 --save-result --result-dir ./result

Test Result

Head 0bb92c89 (main merged up to 21d86ec9), merge base main @ 21d86ec9. vLLM 0.29.0, 1× NVIDIA
L20X, test server 1, .venv-dr2-v029 (CPU) / .venv-fd3-v029 (GPU serving).

Every number below names the commit it was measured at. Sections 0, 1d and 3
were re-run after CI builds 15206, 15224, 15291, 15306 and 15319; section 4's
benchmark was not re-run after the last merge and is labelled accordingly.

0. What CI will actually run

The lanes .buildkite/cuda/test-merge.yml triggers for this PR's files, run
verbatim at 5d17f455:

Gate lane	Command	Result
:29	pytest -sv tests/entrypoints tests/engine -m 'core_model and cpu'	2700 passed, 4 skipped, 74 deselected, 1 xfailed (362.25 s)
:35	pytest -sv tests/ -m 'core_model and cpu' --ignore=diffusion,model_executor,entrypoints,engine	24 collection errors, all missing optional packages in this venv (torchmetrics ×18, mistune ×3, openpyxl ×2, ray ×1); none reference vllm_omni
:131	pytest -s -v tests/e2e/online_serving/test_minicpmo_4_5_duplex.py -m 'advanced_model and cuda' --run-level advanced_model	11 passed (462.53 s)
:23	pytest -sv tests/diffusion -m 'core_model and cpu'	5377 passed, 47 skipped at 014350a8 (two torchmetrics files excluded locally; CI has it). The one real failure, an AST source test that looks for _absorb_diffusion_metrics on Orchestrator after this PR moved it to OrchestratorBase, is fixed in c1dff45d (12 passed)
:44	pytest -sv tests/diffusion/lora/ -m 'advanced_model and cuda'	1 passed at 014350a8
ready · duplex	pytest -s -v test_minicpmo_4_5_duplex.py test_duplex_client_live.py -m 'core_model and cuda'	2 passed at c737b7c0 (test_duplex_client_live_session 1 passed in 777.68 s at 8ae42c6f; section 3 covers the two failures CI build 15206 hit here)
nightly · duplex	pytest -s -v test_minicpmo_4_5_duplex_expansion.py -m 'full_model and cuda'	3 passed at c737b7c0 — admission_and_expiry_reaper, soft_interrupt, server_vad_hard_interrupt (the last two failed in build 15224; section 3)
ready · online (restored)	pytest -s -v tests/e2e/online_serving/test_minicpmo_4_5.py -m 'advanced_model and cuda'	4 passed at 1fbe5013 — the /v1/chat/completions suite this PR had deleted, back unchanged from main (section 3)
:69 entrypoint H100	pytest -s -v tests/entrypoints/ -m 'advanced_model and cuda and H100'	build 15291: 10 passed, 1 failed — the turn-based server-VAD realtime test, deselected in 0a63ceb9 and deferred to RFC PR 5 (section 0 note)
ready · perf	pytest -s -v tests/dfx/perf/scripts/run_benchmark.py --test-config-file tests/dfx/perf/tests/test_minicpmo_4_5_ready.json -m 'core_model and cuda'	1 passed in 745 s at 77f03ddb — Seed-TTS 1×10: 10/10, 4×40: 40/40, 0 failed (build 15306 had 39/40; section 0 note)
ready · perf (3× sample)	same config with num_prompts: [120], max_concurrency: [4]	120/120, 0 failed, 173 s at 6f7ebd25; no turn needed more than the usual 12 s of silence (build 15319 had 39/40; section 0 note)

Lane :35 originally reported 25 errors, one of which was this PR's: a stale
DuplexFence import in tests/e2e/features/fullduplex/test_core_contracts.py,
still pointing at engine.duplex.messages after the symbol moved to
engine.duplex.contracts. Being a collection error it interrupted the whole
lane rather than failing one test. Fixed in 5d17f455.

CI build 15206 failed on four things, all fixed since: the DuplexFence
import above (5d17f455), two AsyncEventResolver imports that aborted lane
:23 at collection so it ran nothing (fixed before 0c6e0747), the protocol
smoke close race (117e7e03) and the live-client hang (8ae42c6f); the
diffusion source test (c1dff45d) was found by re-running lane :23.

CI build 15224 (nightly, at 8ae42c6f) had 10 red jobs: 8 were the agent
being lost mid-run (exit −1, logs stop during model load or at 40–81 % of a
perf run), the duplex lane's test_duplex_server_vad_hard_interrupt (fixed in
c737b7c0, section 3), and the MiniMax-H3 FP8 quality test, which fails
identically on main (its test omits the trust_remote_code that vae.py
requires; nothing on that path is this PR's). The other pipelines at nearby
heads: AMD's one failure was the metrics source test (c1dff45d); Intel's was
No space left on device during git clone; NPU's perf lanes failed at
seed-tts concurrency 8 against max_sessions: 4 — every Seed-TTS utterance
is now a duplex session, so the benchmark probe now queues for a slot on the
retryable resource_exhausted (3dc07acf); NPU's accuracy lane fails a
count check that is independent of this PR (--no-oversample, a 1088-row
Seed-TTS en set, num_prompts=2000; WER itself passed at 0.0208).

CI build 15291 (entrypoint H100 lane, at 93f53f1f) failed one test:
test_server_vad_multi_turn_without_client_commit — a turn-based
Qwen3-Omni session with turn_detection: server_vad and no client commits.
This PR removed the serving-side server VAD that endpointed such sessions, so
a turn-based model now reaches vLLM's realtime handler, which has no server
VAD and rejects the OpenAI-style session.update. That is the P0 raised on
this PR; restoring turn-based server VAD on the transport is RFC PR 5, and
the test is deselected in the lane meanwhile (0a63ceb9) — the file and its
three other realtime tests keep running.

CI build 15319 (ready pipeline at 070b480c, 23 of 26 lanes green) had three
red lanes. Other Test: test_patch.py's fake realtime client did not accept
the until argument stream_silence gained in 77f03ddb (6f7ebd25). Perf:
1/40 Seed-TTS requests timed out waiting for response.done — a model-native
session may choose to listen on the first silence units, and 12 s of silence
then a 180 s wait with no more audio is exactly that stall; the silence budget
is 30 s now, spent only on a turn the model is slow to take since the silence
stops at response.done, a turn needing more than 12 s is logged, and a turn
that never settles is reported with what the model did (6f7ebd25; 3× sample
proof above). Engine&Entrypoints: 16 test_output_metadata_snapshots.py
failures that are main's own — #7608 changed consolidate_tensors to take
the output modality without updating #7448's tests, and main's lane has been
red since build 15307; the tests now pass OutputModality.AUDIO (0bb92c89,
37 passed with test_output_modality.py). The same two failures were AMD's
only red lanes; Intel's was No space left on device during git clone.

CI build 15306 (MiniCPM-o 4.5 perf lane, at 0a63ceb9) failed 1 of 40
Seed-TTS requests at concurrency 4: expected one audio response, got 2.
main drove each Seed-TTS turn with response.create, which yields exactly
one response; the port to the duplex route seeds the text into a model-native
session and streamed the whole 12 s silence cap in real time before waiting
for response.done, so a model that reached turn_eos after a few seconds
kept hearing silence and occasionally spoke again on it. The probe now stops
the silence the moment the turn settles and measures the first response that
carried audio (77f03ddb; a later one is logged as the model's own speech).
Re-run of the same test on the branch server: 50/50 requests, no second
response at all. The same log's touch_duplex_session failed ERROR traceback
was a lease touch racing the session's own close; it is a debug-level result
now.

The two full_model E2E cases that fail (test_duplex_soft_interrupt,
test_duplex_server_vad_hard_interrupt) are not in any gate lane — there is
no full_model lane in test-merge.yml; they run in test-nightly.yml on
H100/B200. Their driver reaches the server through vllm_omni/experimental/.

1. Unittests — refactored duplex framework

742 passed, 0 failed (344.91 s, at 5d17f455).

Suite	Collected	What it drives
tests/engine/duplex/	340+	the real DuplexSessionManager / DuplexSessionRunner / DuplexEngineSession against a fake DuplexStagePort — admission, leases, resume/takeover, close-vs-cleanup ordering, the typed command/event contract, the seeded text turn, and the server-VAD overlap decisions
tests/worker/test_native_duplex_hooks.py	65	worker-side sampling hooks
tests/entrypoints/test_omni_entrypoints.py	64	shared entrypoint bases
tests/clients/	57	DuplexClient, model session configs
tests/benchmarks/	87	benchmark backend and patches
tests/entrypoints/openai_api/test_duplex_api_server.py	21	which routes exist on a duplex server, which are refused, and that chat is wired only when the model declares it
tests/entrypoints/duplex/	17	serving as pure transport (journal, resume, idle timeout, takeover)
tests/engine/test_duplex_orchestrator.py	16	the DuplexOrchestrator, the hierarchy the two surfaces depend on, and replica-death cleanup
tests/entrypoints/openai/test_duplex_session_attachment.py	18	attachment registry, detach generations, resume rollback
tests/entrypoints/test_duplex_omni.py	10	DuplexOmni handle lifecycle
tests/engine/test_duplex_import_boundary.py	10	the turn-based stack never imports duplex code
tests/engine/test_duplex_omni_engine.py	9	the DuplexOmniEngine layer

Cases added by review feedback in this round, each verified to fail when its fix
is reverted:

• test_server_vad_speech_started_reaches_a_barge_in_decision (+2) — server_vad
  with barge_in_on_speech reached an UnboundLocalError on every append
• test_registry_resume_cancelled_mid_delivery_rolls_back_like_a_failure[activation|replay]
  — CancelledError bypassed the resume rollback
• test_a_resume_that_fails_to_activate_does_not_detach_the_live_attachment — a
  losing resume could detach the winner
• test_a_dead_replica_closes_the_sessions_it_was_serving — replica death left
  the owning session and its admission slot alive
• test_raw_wire_hints_cannot_override_the_normalized_typed_fields (+1)
• test_an_item_only_response_create_is_refused_not_left_hanging
• test_the_duplex_stack_extends_the_turn_based_one,
  test_an_unrecognised_message_reaches_the_turn_based_handler

1b. Everything else under tests/engine, tests/entrypoints, tests/clients

2760 passed, 1 failed, 42 skipped, 1 xfailed (401.87 s, at 5d17f455).

The single failure reproduces on main @ 1b6cd282 (2796 passed, 1 failed,
same test), so it is not this PR's:

Failure	Also fails on main
test_stage_engine_core_proc.py::test_preprocess_add_request_preserves_omni_fields	yes

1c. Turn-based regression — AsyncOmni / AsyncOmniEngine / Orchestrator

941 passed, 1 failed (the same pre-existing test above), 39 skipped
(115.28 s, at 5d17f455).

This is the suite that matters for the re-parenting: DuplexOrchestrator now
extends Orchestrator, DuplexOmniEngine extends AsyncOmniEngine, and
DuplexOmni extends AsyncOmni. Covers test_orchestrator.py,
test_orchestrator_error_handling.py, test_async_omni_engine_stage_init.py,
test_stage_engine_core_proc.py, the four test_async_omni*.py files,
test_omni_sleep_mode.py, test_pd_disaggregation.py,
test_stream_finish_reason.py and tests/config/.

The structural half is test_duplex_import_boundary.py, which spawns a fresh
interpreter, imports the turn-based modules and fails if any duplex module
appears in sys.modules. It passes: the dependency runs duplex -> turn-based
and never the reverse.

1d. Diffusion (DiT) lanes — the turn-based path end to end

The CPU suites above never start a DiT stage, so the re-parented
Orchestrator / AsyncOmniEngine / AsyncOmni stack was also run through
CI's diffusion lanes, verbatim, at 014350a8:

Lane	Result
merge gate :23 tests/diffusion cpu	5377 passed, 47 skipped; the one real failure fixed in c1dff45d
merge gate :44 tests/diffusion/lora cuda	1 passed
ready tests/diffusion/cache cuda	8 passed
ready tests/diffusion/batching cuda	6 passed
ready tests/diffusion/offloader cuda	5 passed
ready tests/diffusion/distributed cuda, cards_1	2 passed (297 deselected: CI's L4 marker does not apply on L20X)

accelerate had to be installed in the venv first — without it the cache and
batching lanes fail at pipeline init on both main and this branch — and the
two torchmetrics quality files are excluded locally; CI carries both.

2. Unittests — MiniCPM-o 4.5

515 passed, 2 failed, 3 skipped, 18 deselected (38.50 s, at 5d17f455).

Both failures are test_code2wav_batching.py ([fp32-cache], [bf16-cache])
and reproduce on main @ 1b6cd282 (427 passed, 2 failed, same two ids).
Run in the CPU venv, which does not carry the optional cosyvoice2 package.

3. MiniCPM-o 4.5 E2E and barge-in test

Tier	Result
advanced_model (gate lane :131)	11 passed in 462.53 s, at 1dabb4cd
core_model · test_duplex_client_live_session	1 passed in 777.68 s, at 8ae42c6f
core_model · protocol smoke	1 failed at 1dabb4cd; cause fixed in 117e7e03, regression test in test_duplex_serving.py

Both core-tier failures in CI build 15206 were real, not flakes:

• Protocol smoke (ConnectionClosedError: no close frame received or sent,
  0.63 s after the socket opened): DuplexSessionHandle._deliver marks the
  handle closed in the same step that queues session.closed, and the serving
  read loop exits on that flag — so the endpoint could return, and the ASGI
  server tear the socket down, before the pump had written the terminal event.
  Racy, which is why it passed on re-run. The endpoint now waits for the pump
  to drain a closed session (117e7e03).
• Live client (900 s TimeoutError draining response.audio(); main
  passes the same test in 577 s): ModelChannel._build_stage_output built the
  decision output with from_stage_output(..., finished=True), whose
  source-copy overwrites finished — a resumable stage request never finishes,
  only its segment does — so the MiniCPM-o projector, which recognises a listen
  only on a finished output, projected the model's listen to nothing: no
  response.listen, no next silence unit, no response.done. The trace of a
  failing run: 69 TTS chunks, 64 silence units (the whole auto budget), one
  stage-0 listen, zero projected results, then 600 s of silence until the
  stall net fired. main was covered by its orchestrator queue re-asserting
  finished on every direct-response message. finished is now set after
  construction (8ae42c6f); two harness tests use the real finished=False
  shape and fail with the fix reverted.
• Server-VAD hard interrupt (test_duplex_server_vad_hard_interrupt,
  nightly duplex lane; TimeoutError waiting for the follow-up response,
  main passes in 543 s): the runner's new _maybe_schedule_vad_commit
  committed the turn at the detector's speech_stopped for every session.
  main did that only for turn-based server VAD; a model-native session's
  server VAD was interrupt-only. On a native session the early commit cut
  the utterance short: the model listened on that final unit, the trailing
  silence became a second near-empty turn, and no follow-up response could
  ever appear (like-for-like event streams: main one commit then
  response.created; branch two commits, two listens, silence). The
  auto-commit is skipped for auto-response sessions (c737b7c0); two
  harness tests with a scripted detector pin both contracts.

/v1/chat/completions — restored examples and tests

The route is the ordinary chat service on the duplex engine, so the examples
and tests deleted with the old adapter are back unchanged from main
(1fbe5013, wired into CI again in bcbf17ef). Run against the branch's
duplex server: test_text_to_text_001, test_text_to_audio_with_reference_audio,
test_text_to_audio_with_default_reference, test_mix_to_text_audio_001 —
4 passed (the three audio tests need pyttsx3/espeak-ng and opencc,
which CI carries).

Barge-in

PASS — 4 responses, artifacts in barge_in_out/.

  handshake:  HT_ref_audio.wav            (voice sample only)
  user:       response_required_16k.wav →  response_1_completed.wav  6.40 s
  user:       soft_interrupt_16k.wav      (starts DURING response_1 = barge-in)
                                       →  response_2_completed.wav  7.04 s
                                       →  response_3_completed.wav  3.12 s
                                       →  response_4_completed.wav  2.80 s

/v1/chat/completions on the same server

200 in 2.10 s. The other turn-based routes stay off: /v1/completions
answers 501, /v1/models 200.

Both surfaces exercised at the same time on one process: a websocket duplex
session held open and mid-turn from 0.07 s to 4.31 s, with two chat completions
answering inside that window (3.29 s and 3.81 s) and the session answering
afterwards.

4. E2E benchmark with MiniCPM-o 4.5

Measured at 2bad104c against main @ 1b6cd282 and not re-run since;
the commits added after it are review fixes and tests, none touching the
generation path. Both tables were run back to back in one GPU reservation, same
box, same venv, same dataset and reference audio.

Branch duplex_refactor2 @ 2bad104c

Subset	Requests	E2EL (ms)	TTFT (ms)	TPOT (ms)	Output tok/s	Audio TTFP (ms)	Audio RTF	Generated audio (s)	Responses
1q1a	1/1	308583.36	1.96	10.52	2.51	1.94	0.64	86.88	17
1q1a_math	1/1	303651.53	0.23	10.67	2.55	0.22	0.99	131.00	9
1qna	1/1	303738.58	0.04	8.97	0.62	0.02	0.98	38.84	1

Reference: main @ 1b6cd282

Subset	Requests	E2EL (ms)	TTFT (ms)	TPOT (ms)	Output tok/s	Audio TTFP (ms)	Audio RTF	Generated audio (s)	Responses
1q1a	1/1	309604.28	0.04	10.08	1.20	0.02	0.65	86.88	17
1q1a_math	1/1	305239.48	0.04	9.78	2.25	0.02	1.07	131.00	9
1qna	1/1	305285.54	0.04	8.66	0.60	0.02	0.98	38.84	1

Generated audio and response counts match exactly on all three subsets, which is
the property that matters: the refactor changes who owns the session, not what
the model produces. E2EL is dominated by the ~300 s of input per clip, so the
sub-1 % differences are wall-clock noise. TPOT is within 0.5 ms on two subsets
and 0.9 ms on 1q1a_math, in the branch's disfavour — stated rather than
rounded away, though it sits inside this benchmark's run-to-run spread.

5. Lint and pre-commit

At 1dabb4cd: ruff check clean, ruff format --check clean (2648 files).
The later commits (117e7e03, c1dff45d, 8ae42c6f, c737b7c0, 3dc07acf,
e88f0dab, 77f03ddb, 6f7ebd25, 0bb92c89) each passed both on the files they
touch; the restored files in
1fbe5013 are main's verbatim; pre-commit was not re-run after 1dabb4cd.

pre-commit run --all-files fails 5 hooks — markdownlint-cli2, mypy-3.10,
Ensure test files have CI marks, Lint shell scripts, Check SPDX headers.
All five fail identically on main, and every file they name is untouched by
this PR. Net new hook failures: 0. (typos was the one this PR broke; fixed
in 98c5c0dc.) Lint shell scripts is unknown rather than passing —
shellcheck is not installed on the test box, so it evaluated nothing.

6. Pre-check

precheck-pr, full mode, at 1dabb4cd: 0 blocking, 3 warnings.

Dimension	Result
PR title format	✓ (was ✗ — retitled [Core][Frontend] …)
Examples policy	✓ no new model-specific Python example
Simplification / dead code	✓ (was ⚠ — TurnDetectionConfig.as_realtime removed in a24d8d8f)
SPDX, forbidden imports, new torch.cuda sites	✓ / 0 / 0
Test CI marks	✓ every added test file carries a level and hardware mark
Code quality	⚠ 70 broad except Exception added; 60 of 81 log, re-raise, return or emit within two lines. Worth a narrowing pass later
PR description integrity	⚠ resolved by this update
Pre-commit gates	⚠ see section 5 — 0 net new

BEFORE SUBMITTING: read CONTRIBUTING.md and run the precheck-pr skill with the code agent for a self-check against project conventions.
(anything written below this line will be removed by GitHub Actions)

[vllm-omni-review-bot]

This PR appears to belong to: docs/design/module/observability.md, docs/design/module/engine_orchestration.md.

Module owners: @lishunyang12 @vraiti @tzhouam

Routing: @lishunyang12 via module of the changed files; @vraiti via module of the changed files; @tzhouam via semantic router, CODEOWNERS

@chickeyton, please review your own changes and leave a short self-review comment describing what you checked. PRs without author self-review may not be assigned a reviewer.

Please take a look when you have a chance. If you would like an automated review, mention @vllm-omni-review-bot in a comment.