[merceod]

Why

--rust-frontend had no port of Qwen3TTSAdapter, so mstar-server answered 404 on /v1/audio/speech for every Qwen3-TTS model (it logged Qwen3TTSAdapter has no port in this frontend) and all TTS serving and benchmarking ran on Uvicorn. This ports it, along with the parts of the Python speech route a port needs to answer the same way (format check order, sentence chunking of long inputs), and measures what the Rust frontend buys on the speech lane.

What changed

• rust/server/src/adapters.rs: Adapter::Qwen3TTS (from_python("Qwen3TTSAdapter"), supports_speech). speech_to_request builds what Qwen3TTSAdapter.speech_to_request builds: extra fields pass through, voice set when sent, instructions becomes instruct (an explicit instruct wins, an empty one is dropped), temperature / top_p / seed onto the Talker keys, a sent speed forwarded so the server logs it as ignored. speech_chunking() holds each adapter's speech_chunk_* settings (Qwen3-TTS splits from 600 characters, the others only on sentence_chunking: true).
• Same file: Orpheus and Qwen3-Omni now forward a sent speed too. Their Python adapters already do (_forward_sent_speed); the Rust ports dropped it.
• rust/server/src/speech_chunking.rs (new): split_sentences, _plan_chunks and _chunk_kwargs ported. Lengths count characters and whitespace is Python's str.isspace, so the pieces are Python's.
• rust/server/src/main.rs audio_speech: create_speech's order: format check, adapter, piece plan, then one submit per piece (<id>-<i> ids, seed + i per piece, two pieces submitted ahead of the one playing). A stream still waits for the first piece's first item before committing the 200, then writes the WAV header and one HTTP chunk per audio chunk, as _stream_pcm does; a later error aborts the body, and dropping an unread piece aborts it.
• rust/server/src/media.rs check_audio_format: an empty response_format means wav, and a stream that asks for a container gets the Python text (response_format 'mp3' cannot be streamed by this server; use one of: wav, pcm). A non-streamed mp3/flac/opus keeps the existing Rust refusal, since there is no encoder here.
• rust/server/src/protocol.rs: SpeechRequest.speed tells unsent from null, as pydantic's model_fields_set does.
• rust/server/Cargo.toml: regex, for \p{M} (the splitter never starts a piece on a combining mark) and Python's printable classes in a quoted refusal. Already in the lock through tokenizers (1.13.0); no new crate.
• docs/environment_variables.rst: ref_audio under "Rust frontend limitations".

ref_audio (Qwen3-TTS Base voice cloning) is a 400 on the Rust frontend that points at the Python frontend: the Python route decodes the clip and checks its length before submit, and mstar-server has no audio decoder.

Tests

• test/rust/test_rust_speech.py (new, 70 cases; runs in the rust-transport job, which installs fastapi): the real mstar-server and bridge against Python's create_speech, on stubs that answer every piece with the same audio. For Orpheus, Qwen3-Omni and Qwen3-TTS, 9 requests x stream on/off: the submits (text, modalities, model kwargs, piece ids) and the status, content type, cache-control and body bytes are equal. 8 refusals (stream + container, too many pieces, an engine refusal before and after the first piece, a model without speech) give the same status and JSON on both. A streamed WAV and PCM, one piece and three, are the same list of HTTP chunks as the Python body yields (the 44-byte header, then one chunk per audio chunk). 120 random long inputs over the splitter's edge cases (closers, CJK ends, paragraph breaks, clause marks, runs without spaces, combining marks, U+001C, NBSP) give the same 1898 piece texts and the same 8 piece-count refusals. Against main's binary, 59 of the 70 fail.
• cargo test --release in rust/server: 76 passed (17 new: the splitter, the format refusal and Python repr, the Qwen3-TTS mapping, speed presence).
• pytest test/rust: 591 passed, 14 skipped (main: 521 passed, 14 skipped; the same skips). test_rust_openai_adapters.py no longer lists Qwen3-TTS speech as not ported.
• ruff check .: clean. api_server modules in test/modular (openai adapters/router/realtime/transcriptions, speech chunking, media, serving metrics, body limit, request ids): 1124 passed, 2 failed on both this branch and main, the same two ids.
• CPU Core paths (test/modular test/distributed test/chatterbox test/kokoro test/omnivoice test/sampling_test mstar/model/*/tests benchmark/worker_phases, -n 6): 11220 passed, 1025 skipped, 20 failed, the same 20 ids on main (eval_compare_multigpu/transport, send_registration_failure, wan_vae / z_image bit-exact, lingbot, serving_metrics, client_error_text); the TP lockstep sim (11) and piecewise signature (16) modules pass on both.
• These ran on 6583ca1. The pushed head merges main a7fa870 (Qwen3-Omni: one grouping for the prefill schedule and the KV layout #1165, Qwen3-Omni only, nothing under rust/, mstar/api_server or test/rust); on the merge, the three test/rust frontend modules (172 passed, 2 skipped) and ruff check . were rerun.

Gate

2 x B300 (sm_103), driver 595.91.07 (CUDA 13.2), torch 2.12.1+cu132, FlashInfer 0.7.2.post1; Xeon 6740E host, server pinned to 8 cores and the client to 4, with idle-priority spinners on them so the cores don't park. Qwen3-TTS 1.7B, stock configs/qwen3tts_1p7b.yaml, -O2, GPU 0. Seed-TTS EN, first 200 sentences, voice ryan, closed loop, streamed WAV; per c two passes, the first after 5 x c warmup requests. Four sessions alternated, each a fresh server and Inductor cache: main (482dfc7) on the Python frontend, then this branch (6583ca1) on the Rust frontend, twice. Mean of a session's two passes:

c	session	audio-s/s	RTF p50	first byte p50/p95 ms	audible p50/p95 ms	e2e p50/p95 s	WER %	API CPU / request, ms (utime, +stime)
1	Python s1	19.9 (19.9-20.0)	0.050	31/32	31/32	0.22/0.37	1.04	6.24 / 8.14
1	Rust s1	19.7 (19.7-19.7)	0.051	30/32	30/32	0.22/0.38	1.61	6.44 / 9.71
1	Python s2	20.1 (20.1-20.2)	0.050	31/32	31/32	0.21/0.37	1.28	6.25 / 8.12
1	Rust s2	20.0 (19.9-20.0)	0.050	30/32	30/32	0.21/0.38	1.26	6.25 / 9.44
8	Python s1	96.5 (96.1-97.0)	0.082	51/63	51/63	0.36/0.57	1.76	5.33 / 7.07
8	Rust s1	97.0 (93.9-100.2)	0.080	49/60	49/60	0.34/0.53	2.42	4.60 / 6.96
8	Python s2	99.0 (98.3-99.7)	0.080	50/61	50/61	0.33/0.60	1.95	5.37 / 6.96
8	Rust s2	97.2 (96.9-97.5)	0.081	50/62	50/62	0.34/0.56	2.33	4.87 / 7.34
32	Python s1	216.1 (216.1-216.2)	0.141	85/168	85/168	0.58/1.01	1.45	4.79 / 6.21
32	Rust s1	216.4 (214.4-218.5)	0.141	84/136	84/136	0.60/1.04	2.47	4.21 / 6.32
32	Python s2	216.0 (209.4-222.6)	0.140	85/173	85/173	0.58/0.95	1.28	4.71 / 6.14
32	Rust s2	214.5 (212.7-216.3)	0.140	83/136	83/136	0.60/1.05	1.28	4.18 / 6.42

• Throughput, RTF and TTFA p50 are level at all three concurrencies. First-byte p95 at c=32 is lower on the Rust frontend in both sessions, 168/173 ms against 136/136 ms; e2e p50 at c=32 is 0.58 s on both Python sessions and 0.60 s on both Rust ones.
• The lane is not API-bound here: the API processes (the Python API server, plus mstar-server on the Rust frontend) total 30% of one core at c=32 on either frontend. On the Python frontend that is the Uvicorn loop 12-13%, the data worker's preprocess thread 11% and the message thread 5%; on the Rust frontend the preprocess thread 11%, the bridge loop 6%, the message thread 5% and mstar-server 5-6%. Over each arm's four passes, utime per request drops 12% at c=8 and c=32 (5.35 → 4.73 ms, 4.75 → 4.19 ms) but utime + stime does not (7.02 → 7.15, 6.17 → 6.37 ms), and at c=1 utime + stime is 18% higher (8.13 → 9.57 ms, mostly stime; not traced).
• Hypothesis, not traced: the lower first-byte tail at c=32 is request handling no longer queueing behind the Uvicorn loop.
• WER (Whisper large-v3) passes on all 24 passes, 1.00-3.70%. Length runaways (> 2.5x a sentence's median audio): 15 of 2400 requests on main, 18 of 2400 here, the rate voice ryan has on main.
• Seeded bytes: on one shared warm Inductor/Triton cache, five servers on GPU 1, alternated Rust, Python, Rust, Python, Rust, returned byte-identical bodies (WAV header and PCM) for all 40 seeded requests (seed 1234, the first 40 sentences, c=1). The boot that filled the cache (Python frontend) differs from all five on every sentence; compiling boots of this config have differed from warm ones before, on main too.

Past the gate, one more session per arm at c=64 and c=128 (main first, not alternated; each pass is about 3 s, 200 requests being 1.5 to 3 rounds at these concurrencies, so audio-s/s there is mostly ramp and tail):

c	frontend	audio-s/s pass 1, 2	RTF p50	first byte p50/p95 ms	e2e p50/p95 s	API CPU / request, ms	API busy % of a core
64	Python	281.4, 275.2	0.207	127/284	0.84/1.53	4.44 (5.73)	33
64	Rust	282.3, 238.6	0.208	133/251	0.82/1.43	3.81 (5.69)	30
128	Python	310.6, 305.3	0.327	517/579	1.28/2.21	4.14 (5.28)	34
128	Rust	306.9, 310.6	0.327	477/552	1.28/2.14	3.68 (5.27)	37

The Rust c=64 pass 2 took 3.7 s against 3.1 to 3.3 s for the other three with no request over 1.9 s; not traced. Even at c=128 the API path stays near a third of one core on either frontend (the Uvicorn loop 17%), while first byte grows to about 0.5 s on both; so the API path is not what limits the speech lane through c=128, and the Rust frontend buys no throughput on it. Hypothesis, not checked: that first byte is requests waiting for the engine.

Not in this PR

• ref_audio (voice cloning) on the Rust frontend: it needs the clip decoded and length-checked before submit, as _resolve_ref_audio / check_ref_audio do.
• The other speech adapters without a port (OmniVoice, Kokoro, Chatterbox, FishS2Pro, CosyVoice3, VoxCPM2). None has Qwen3-TTS's contract: OmniVoice and Kokoro always forward speed, Kokoro, OmniVoice and VoxCPM2 map no temperature / top_p, OmniVoice, Chatterbox, Fish, CosyVoice3 and VoxCPM2 take a ref_audio clip, CosyVoice3 splits with its own rules. Fish (44.1 kHz) and VoxCPM2 (48 kHz) also need the launcher to pass the model's output rate; mstar-server stamps MSTAR_SAMPLE_RATE, default 24000.
• GET /v1/audio/voices on the Rust frontend.
• temperature / top_p sent as strings still reach the API server as sent (the API server's normalize_generation_kwargs reads them), where the Python frontend coerces them to floats first; unchanged here.