[elpsykongloo]

Purpose

Add native Lychee-FD serving through Unified Duplex and MRV2. AR produces separate semantic, codec and control channels; native Token2Wav produces ordered 24 kHz PCM with hop10/lookahead3 and natural-final flushing. Cancellation, speech EOS and slot reuse retain request/epoch ownership. This also adds the generic append/recovery and output-lifetime support needed by that path.

C1 eager single-GPU serving is the default, with explicit C2/C4 profiles. Configuration aliases, weight mapping, deployment profiles, model documentation and a public-model recipe are included. Migration-only oracles and tuning tools are excluded. Third-party decoder source retains its license notices.

Design: #8634

Validation

• Final head: ce737762, based on upstream 4e675f8b, with DCO Signed-off-by.
• Final CPU regression: 1425 passed, 2 skipped, 1 deselected, 78.72 seconds. Includes affected core tests and the current upstream Duplex API regression.
• Applicable import/CUDA-helper/SPDX/test-mark/Ruff checks, upstream mypy scope and documentation checks: passed. Real fresh-environment libdevice imports and prompt Mel dependency regressions: passed.
• Final source image: built; CPU runtime imports and pip check passed.
• Earlier migration image on A100 80 GiB: 327 committed windows; ordinary C2/C4 admission, cancellation, barge-in and reopen; actual WebSocket resume/replay/token rotation/playback ACK; codec/Mel/chunk/final/sample-clock boundaries. Instrumented fault cases are separate correctness evidence.
• Standalone Token2Wav steady run: 1803 seconds, 64 measured cycles, 192 natural finals, stable live memory. Scope is one recorded codec fixture, not whole-service acceptance.

Still running: final-head standard serve --omni, complete-response capture and whole-service steady validation. A previous whole-service window/collector timeout is unresolved and is not counted as a pass. This PR remains a Draft while these results are completed.

Not run: exclusive realtime RTF/latency A/B while other GPU tasks occupy the server; public-checkpoint GPU/numerical equivalence; additional GPU architectures, CUDA Graph, overlap and multi-GPU qualification. No speedup or realtime SLA is claimed. Public configuration/index layout is checked, while current GPU evidence uses the local released checkpoint.

Reproduction

See docs/models/lychee_fd.md and recipes/HIT-TMG/Lychee-FD-A100.md for model downloads, separate Token2Wav/prompt assets and the standard server command. Validation runtime: NVIDIA A100 80 GiB, vLLM/vLLM-Omni 0.31, Torch 2.13.0+cu132, cuDNN 92000, torchaudio 2.11; TP=PP=1 and eager. Shared functional runs use an explicit reduced-memory profile and do not qualify the default profile's realtime behavior.

CPU regression from this branch:

export CUDA_VISIBLE_DEVICES=''
export OMP_NUM_THREADS=2 OPENBLAS_NUM_THREADS=2
mapfile -t changed_tests < <(git diff --name-only 4e675f8b HEAD -- tests | sed -n '/\.py$/p')
python -m pytest -q "${changed_tests[@]}" \
  tests/entrypoints/openai_api/test_duplex_api_server.py \
  -m 'not cuda' --run-level core_model

Initial scope excludes text-only/image/chat-completions, rollback/audio truncation and multi-GPU. Resume is process-local. Optional compatibility math falls back outside its qualified runtime or when the default libdevice is absent; explicit invalid overrides and real kernel failures remain errors.

AI assistance

Codex and its subagents assisted code, tests, documentation and validation. The results above distinguish final-head checks from historical evidence and tests not run. Contributor and maintainer review remain necessary; this Draft does not assert completed human review.