tts-cpp: supertonic Engine streaming via multilingual chunker + callback

[ogad-tether]

Summary

Adds native streaming synthesis to the Supertonic Engine mirroring the chatterbox StreamCallback API. Splits text into chunks via a multilingual splitter, runs the full per-chunk pipeline on the resident model, and invokes a user callback synchronously with each chunk's PCM as it's produced. The returned SynthesisResult.pcm still contains the concatenated audio so existing batch callers are unaffected — the callback is an addition, not a replacement.

Based on supertonic_optimizations (which now includes #15's Metal port and #21's CPU regression fix).

Why this is chunked-pipeline streaming, not token-streamed-inside-one-utterance

Chatterbox achieves true token-level streaming because its model and training were designed for it: T3 emits audio-rate speech tokens, S3Gen uses causal convs, HiFT carries mel-frame cache + F0 phase across chunks, and the encoder lookahead trim cleans boundary effects.

Supertonic has none of that infrastructure. Single-stage pipeline, bidirectional attention over the full latent, per-utterance duration prediction, no cache continuity across forward passes, and trained on full sentences only. Forcing causal attention or chunked input at inference time produces audio the model never learned to generate (verified — sub-30-token stubs glitch on dropped/muddled phonemes regardless of preprocess tweaks).

So this PR ships what's actually achievable: sentence-aligned chunks for multi-sentence input (acoustically equivalent to batch), mid-clause/whitespace chunks for long single-sentence input where there's no other choice, and a is_continuation preprocess flag so the model isn't told "this is a complete sentence" when it isn't. Inter-chunk pauses and rate shifts at non-sentence seams are inherent to per-chunk synthesis on a non-streaming-trained model and can't be fixed at this layer.

What ships

src/supertonic_chunker.{h,cpp} — Unicode-aware multilingual splitter

• Two-window boundary search:
  • Sentence-end search: [target/2, 2*target] (wide). Catches long-but-reasonable first sentences in multi-sentence input but narrow enough that genuinely runaway sentences (>2× target without internal periods) fall through to whitespace so they still stream rather than dumping the whole tail as one chunk.
  • Clause / whitespace fallback: [target ± tolerance_pct]. User-controlled.
• Punctuation tables: ASCII .?!, CJK 。？！, Devanagari ।॥, Urdu ۔, double ‼⁇⁈⁉ for sentences; ASCII / fullwidth / Arabic comma, semicolon, colon, closing brackets for clauses.
• Whitespace fallback covers CJK / Thai / Lao / Khmer where punctuation may be absent or sparse.
• stream_min_chunk_tokens (default 30) is a hard floor. Below that the model emits dropped/muddled phonemes on stub input. Effective targets are max(target, min). The sentence/clause/whitespace search lower bound is clamped to start + min so the chunker never proactively aims for a sub-minimum chunk.
• Tail-merge uses the chatterbox heuristic max(6, target/3) (16 for target=50), NOT the min_chunk floor — using min_chunk would swallow a complete final sentence on info-dense languages (e.g. Korean "공원에서 산책하기 좋은 날이다." is 18 cps — below the 30-cp floor — but is a perfectly valid sentence-aligned chunk).
• Shared terminator table. The is_sentence_end_cp(uint32_t) predicate is declared in supertonic_chunker.h and reused by the engine's per-chunk continuation detector — additions to the terminator set (Ethiopic ።, Tibetan ། …) live in exactly one place.

is_continuation flag through preprocess

• supertonic_preprocess_text and supertonic_text_to_ids accept an is_continuation bool. When true, the auto-appended terminal period is skipped — used by streaming for chunks that don't end on a natural sentence terminator. Avoids the original "park.K" trailing-phoneme bug where the model spoke a stub chunk as a complete sentence with falling intonation + tail artifacts.

Engine streaming path

• Single opts.seed for every chunk (no per-chunk perturbation; different chunks have different latent_len so noise tensors differ even with the same seed). Earlier opts.seed + k perturbation occasionally landed chunks on glitchy nearby seeds.
• Per-chunk is_continuation derived automatically by checking the chunk's trailing code point against the shared sentence-terminator set (ASCII + CJK + Devanagari + Urdu).
• 10 ms raised-cosine anti-click fade on inter-chunk seams only. First chunk start and last chunk end stay untouched so streamed output is acoustically equivalent to batch at the endpoints.

Engine API additions

using StreamCallback = std::function<void(
    const float * pcm, std::size_t samples, int chunk_index, bool is_last)>;

struct EngineOptions {
    // ...
    int stream_chunk_tokens        = 0;
    int stream_first_chunk_tokens  = 0;
    int stream_chunk_tolerance_pct = 20;
    int stream_min_chunk_tokens    = 30;
};

SynthesisResult synthesize(const std::string & text,
                           const StreamCallback & on_chunk);

supertonic-cli flags

• --stream-chunk-tokens N — target chunk size in text tokens (0 disables; 50 ≈ 1-3 s English audio)
• --stream-first-chunk-tokens N — smaller first-chunk override for first-audio latency
• --stream-chunk-tolerance-pct N — clause/whitespace boundary-snap window
• --stream-min-chunk-tokens N — hard floor on chunk size (default 30)
• --out - streams raw s16le PCM on stdout (one buffered fwrite per chunk). Pipe into ffplay -f s16le -ar 44100 -ch_layout mono -i - or sox -t raw -b 16 -e signed-integer -r 44100 -c 1 - -d.
• SUPERTONIC_LOG_CHUNKS=1 logs chunker boundaries AND per-chunk is_continuation; SUPERTONIC_DUMP_CHUNK_WAVS_PREFIX=path- dumps per-chunk WAVs for debugging.

Test plan

• Build: cmake --build tts-cpp/build --target supertonic-cli -j 8
• Batch vs streaming acoustic equivalence on 3-sentence text: streamed 10.07 s vs batch 10.11 s, no audible difference at chunk seams or endpoints
• First-audio latency on 6-sentence 18 s paragraph: first audio in ~1 s vs batch's ~4–5 s synth-then-play
• Long run-on single sentence (245 chars, no internal .?!): chunker produces 5 evenly-sized chunks (30, 52, 54, 52, 55 cps) via whitespace fallback. All chunks above the min floor.
• Multilingual run-on stress (same 245-ish char run-on in en/fr/pt/ko):
  • EN: 5 chunks, FR: 6 chunks, PT: 5 chunks, KO: 3 chunks (info-dense Hangul → fewer chunks at same cp target).
• Multilingual standard 3-sentence: all four languages chunk at sentence boundaries (. for en/fr/pt, . and CJK 。 recognized for ko).
• Stdout streaming via ffplay: chunk-by-chunk timing in stderr proves later chunks synthesize during earlier chunks' playback.
• CPU streaming (post-QVAC-18966 [TTS GGML] Fix CPU regression #21 rebase): 3-sentence English at --n-gpu-layers 0 writes 10.07 s WAV in 1.09 s wall-time (~9× realtime), exit code 0, no abort. Stdout streaming on CPU produces byte-exact output (samples × 2). Multilingual sentence detection on CPU produces same is_continuation=0/1 flags as Metal (en/fr/pt/ko verified — engine and chunker agree on Unicode terminator predicate).
• Validation script at /tmp/supertonic-validate-review-fixes.sh covers the de-dup correctness (incl. CJK 。 decode through engine → shared is_sentence_end_cp) and stdout-byte-exactness — 4/4 PASS.
• Empirical regressions caught during iteration:
  • Per-chunk seed perturbation (opts.seed + k) occasionally landed on glitchy nearby seeds → now uses opts.seed everywhere.
  • 100 ms tail fade on the last chunk faded legitimate final words → reverted to uniform 10 ms after the seed fix.
  • Tail-merge at min_chunk_tokens swallowed valid CJK trailing sentences → relaxed to max(6, target/3).
  • 3× sentence-search window slurped runaway-sentence tails as one chunk → tightened to 2×.
  • Duplicate sentence-terminator table between engine + chunker → shared is_sentence_end_cp via header.
  • Per-sample fwrite in stream_emit_pcm_stdout → buffered single fwrite per chunk.

Known limitations

• The pre-existing ggml-metal residency-set assertion at process exit (ggml-metal-device.m:612) fires after every Metal-backed run on this branch and on master. Unrelated to streaming; audio is written before exit.
• CJK languages other than Korean aren't accepted by the current Supertonic model GGUF (--language zh rejected at preprocess; supported set: en, ko, es, pt, fr). Chunker handles CJK punctuation correctly; no model support yet.
• Mid-sentence streaming has audible seam artifacts (small pauses + rate shifts) in all languages. This is architectural — the duration predictor and attention run per-chunk on a non-streaming-trained model. The continuation flag avoids the artificial-period failure mode but can't share prosody across chunks. The API we ship carries forward unchanged if/when a streaming-trained Supertonic appears — only synthesize_streaming's internals would change.

Files

Type	Path
Edit	tts-cpp/include/tts-cpp/supertonic/engine.h
Edit	tts-cpp/src/supertonic_engine.cpp
Edit	tts-cpp/src/supertonic_cli.cpp
Edit	tts-cpp/src/supertonic_preprocess.cpp
Edit	tts-cpp/src/supertonic_internal.h
Edit	tts-cpp/CMakeLists.txt
New	tts-cpp/src/supertonic_chunker.h
New	tts-cpp/src/supertonic_chunker.cpp

🤖 Generated with Claude Code

[GustavoA1604]

Cursor flagged these two things, could you check if they make sense?

Duplicated sentence-terminator code point table (supertonic_engine.cpp + supertonic_chunker.cpp) — chunk_ends_with_sentence_term() in the engine and is_sentence_end_cp() in the chunker are identical switch statements. If the set grows (e.g. Ethiopic ።, Tibetan །) only one copy will be updated. The fix is to un-anonymize is_sentence_end_cp in the chunker and have the engine call it.

stream_emit_pcm_stdout writes one sample at a time (supertonic_cli.cpp) — It calls fwrite(&v, 2, 1, stdout) in a per-sample loop, meaning ~44k–132k syscall-adjacent calls per chunk. Should build a std::vector<int16_t> and write the whole buffer in one fwrite.

[ogad-tether]

Both fixes are in 72b9d561. Thanks for catching these — both were real issues.

1. De-duplicated terminator table. is_sentence_end_cp() is now declared in supertonic_chunker.h and called from the engine's chunk_ends_with_sentence_term(). The engine still owns the UTF-8 trim/decode (its input is a std::string, not a raw code point), but the predicate + multilingual table live in exactly one place. Future additions (Ethiopic ።, Tibetan ።, etc.) now need one edit instead of two — and the older Cursor-flagged code path can't drift out of sync.

2. Buffered stdout write. stream_emit_pcm_stdout now builds the chunk's int16_t buffer once and writes it with a single fwrite, then flushes. No semantic change to the bytes on stdout — verified post-fix: 59487 samples produces exactly 118974 stdout bytes (samples × 2), same as before. Just removes the ~44k–132k tiny fwrite calls per chunk.

Both verified on smoke: multi-sentence chunker still gives 3 sentence-aligned chunks (no regression in the predicate), stdout streaming byte count is exact.