feat: add MOSS-TTS-Nano single-stage TTS support

[linyueqian]

Summary

• Add single-stage MOSS-TTS-Nano (0.1B) TTS model support with online serving via `/v1/audio/speech`
• Integrates OpenMOSS-Team/MOSS-TTS-Nano AR LM + MOSS-Audio-Tokenizer-Nano codec
• 15 built-in voice presets (6 ZH, 4 EN, 5 JA), 48 kHz stereo output
• Gradio demo with AudioWorklet gapless streaming player and TTFP/RTF metrics
• Progressive streaming via AR runner — TTFP reduced from ~3.1s to ~0.11s (30x improvement)

Doc/skill updates split into #2806.

Changes

Model integration:

• `vllm_omni/model_executor/models/moss_tts_nano/` — model wrapper using `trust_remote_code` for upstream HF classes; VoxCPM-style per-request `inference_stream()` generator stored in `_stream_gens`; each `forward()` call pops one chunk via `next(generator)`
• `vllm_omni/model_executor/stage_configs/moss_tts_nano.yaml` — single-stage config (`worker_type: ar`, `is_comprehension: true`, `async_chunk: false`, `max_num_seqs: 4`)
• `vllm_omni/model_executor/models/registry.py` — register `MossTTSNanoForCausalLM`

Serving layer:

• `vllm_omni/entrypoints/openai/serving_speech.py` — add `_MOSS_TTS_MODEL_STAGES` constant, extend `_TTS_MODEL_STAGES` union, add `moss_tts_nano` branch in `_get_tts_model_type()`, add `_validate_moss_tts_request()` and `_build_moss_tts_params()` methods

Streaming (AR runner):

• Switched from `GPUGenerationWorker` to AR worker with `OmniARScheduler`, using the VoxCPM-style generator pattern
• `inference_stream()` stored per-request; each `forward()` call yields one audio chunk
• `compute_logits()` emits EOS only when the last chunk is yielded; AR scheduler loops until EOS for progressive audio output

Examples:

• `examples/offline_inference/moss_tts_nano/` — CLI end-to-end script + README
• `examples/online_serving/moss_tts_nano/` — Gradio demo, server/demo launch scripts, README

Tests:

• `tests/e2e/offline_inference/test_moss_tts_nano.py` — English, Chinese, deterministic, batch, voice presets
• `tests/e2e/online_serving/test_moss_tts_nano.py` — non-streaming WAV, streaming PCM, Chinese
• `.buildkite/test-merge.yml` — pre-merge CI entry

Test plan

• Offline inference verified on H20 (English/Chinese/Japanese voices)
• Online serving verified via curl (`/v1/audio/speech` returns 48 kHz audio)
• Gradio demo verified via SSH tunnel (AudioWorklet streaming player)
• Progressive streaming verified: TTFP ~0.11s (vs ~3.1s before)
• Pre-commit hooks pass (ruff F841 fix, ruff format applied)
• DCO sign-off email matches git author (`linyueqian@outlook.com`)
• Buildkite pre-merge CI (L4 GPU)

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

[hsliuustc0106]

BLOCKER scan:

This PR has a merge conflict. Please resolve the conflict before proceeding with review.

OVERALL: MERGE CONFLICT

VERDICT: REQUEST_CHANGES

[hsliuustc0106]

I think you can update the skills to allow community users to contribute more effectively

[linyueqian]

I think you can update the skills to allow community users to contribute more effectively

will do

[lishunyang12]

Code Review: feat: add MOSS-TTS-Nano single-stage TTS support

Overall this is a well-structured PR that follows existing patterns in the codebase (VoxCPM-style generator, stage config YAML, registry entry, serving layer extension). The examples, tests, and documentation are thorough. A few issues to address before merging:

---

Issues

1. _build_moss_tts_params does not handle ref_audio (serving layer bug)
vllm_omni/entrypoints/openai/serving_speech.py — _build_moss_tts_params() maps ref_text to prompt_text but completely ignores request.ref_audio. The online serving README documents custom voice cloning via ref_audio as a supported feature, and the model's _create_stream_gen() reads prompt_audio_path from additional_information. Without wiring ref_audio through the serving layer, voice cloning via the /v1/audio/speech endpoint will silently fail to use the reference audio.

You need to resolve ref_audio (download/decode the data URL) and pass the path through as prompt_audio_path in the params dict, similar to how CosyVoice3 or VoxCPM2 handle it.

2. _ar_emit_stop_token is shared mutable state across a batch — race condition
modeling_moss_tts_nano.py — self._ar_emit_stop_token is a single boolean on the model instance. In forward(), it is set to all(last_chunk_flags) — meaning if any request in the batch is still generating, all requests get non-EOS logits from compute_logits(). This means finished requests are kept alive (emitting empty audio) until the slowest request in the batch completes. For max_num_seqs: 4 this could be significant. Consider making this per-request (e.g., a dict keyed by request ID) so the AR scheduler can finish individual requests independently.

3. _create_stream_gen buffers all chunks then yields — "streaming" is misleading
The docstring says "yields one audio chunk per forward() call" for progressive streaming, and the PR description claims "TTFP reduced from ~3.1s to ~0.11s." However, looking at the generator, it calls self._lm.inference_stream() which does yield events progressively, and those are yielded out. This part looks correct on re-read. But the comment at line ~1949 says "We buffer first because inference_stream mixes audio events with a final result event" — this comment is outdated/misleading since you actually yield chunk, False inside the loop. Please clean up the comment to match the actual streaming behavior.

4. torch.manual_seed() in _create_stream_gen sets global RNG state
modeling_moss_tts_nano.py line ~1925 — calling torch.manual_seed(seed) and torch.cuda.manual_seed_all(seed) sets the global RNG state. In a concurrent batch with max_num_seqs: 4, one request's seed will overwrite another's. Consider using a torch.Generator for per-request determinism, or at least document that seeding is best-effort and not safe under concurrency.

5. _stream_gens dict is not thread-safe
self._stream_gens is a plain dict mutated without the existing self._lock. While the AR worker is likely single-threaded, the _lock is already used for load_weights, so if there's any possibility of concurrent forward calls (e.g., from async scheduling), this could corrupt state. Either document the single-thread assumption or protect mutations with the lock.

6. Offline test _collect_audio has a typo: AssertionError
tests/e2e/offline_inference/test_moss_tts_nano.py line 1331 — raise AssertionError is misspelled as AssertionError. This would actually raise a NameError at runtime instead of the intended assertion.

---

Minor / Nits

• ## MOSS-TTS-Nano comment in registry.py — other entries don't use ## headers. Use # for consistency.
• for _ in weights: pass in load_weights() — add a comment explaining this drains the iterator to satisfy the vLLM weight-loading protocol, since it's non-obvious.
• Gradio demo is 690 lines — the inline AudioWorklet JS and HTML templates are large. Consider extracting them to separate files under the example directory for maintainability (not blocking).
• _DEFAULT_MODE = "continuation" vs online serving README says default voice is "Junhao" which implies voice_clone mode — the defaults are inconsistent between offline and online paths. The offline example uses voice_clone as default mode while the model code uses continuation. Clarify which is intended.
• CI step uses gpu_1_queue — confirm this is the right queue for L4 GPUs, as the test decorator specifies res={"cuda": "L4"}.
• _REPO_ROOT calculation in end2end.py uses Path(__file__).resolve().parents[4] — this is fragile and breaks if the file is moved. Consider using a more robust path resolution or accepting it as a required CLI arg.

---

What looks good

• Clean single-stage architecture with well-documented YAML config
• Proper use of the VoxCPM generator pattern for streaming
• Good test coverage (offline: English, Chinese, deterministic, batch, voice presets; online: WAV, streaming PCM, Chinese)
• Serving layer integration follows existing patterns cleanly
• Gradio demo with AudioWorklet streaming is a nice addition

Please address items 1, 2, 4, and 6 before merging. Items 3 and 5 are lower priority but worth fixing.

[linyueqian]

@Sy0307 PTAL.

[Sy0307]

Overall LGTM, one minor issue:

_stream_gens leaks on abnormal request termination

When a request is cancelled, timed out, or preempted, the AR scheduler notifies the model via on_requests_finished() (gpu_ar_model_runner.py:324, guarded by hasattr). This model doesn't implement that method, so terminated requests leave their generator permanently in _stream_gens — forward() never visits it again, GC can't collect it (dict holds a reference), and the finally block never runs (temp files leak too).

Suggest adding cleanup following the voxcpm2_talker.py pattern (on_requests_finished + _deferred_cleanup_ids):

def on_requests_finished(self, finished_req_ids: set[str] | list[str]) -> None:
    for req_id in finished_req_ids:
        gen = self._stream_gens.pop(str(req_id), None)
        if gen is not None:
            gen.close()