Skip to content

Export KittenTTS models - #3604

Merged
csukuangfj merged 1 commit into
k2-fsa:masterfrom
csukuangfj:update-kitten
May 12, 2026
Merged

csukuangfj merged 1 commit into
k2-fsa:masterfrom
csukuangfj:update-kitten

Conversation

@csukuangfj

@csukuangfj csukuangfj commented May 12, 2026 •

Copy link
Copy Markdown
Collaborator

See also #3591
cc @dewana-sl

You can find the exported models at https://github.com/k2-fsa/sherpa-onnx/releases/tag/tts-models

Screenshot 2026-05-12 at 14 55 08

Usage

wget https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/kitten-micro-en-v0_8.tar.bz2
tar xf kitten-micro-en-v0_8.tar.bz2
ls -lh kitten-micro-en-v0_8/
total 87792
drwxr-xr-x@ 122 fangjun  staff   3.8K 28 Nov  2023 espeak-ng-data
-rw-r--r--@   1 fangjun  staff    11K 12 May 14:53 LICENSE
-rw-r--r--@   1 fangjun  staff    39M 12 May 14:53 model.onnx
-rw-r--r--@   1 fangjun  staff   796B 12 May 14:53 README.md
-rw-r--r--@   1 fangjun  staff   1.0K 12 May 14:53 tokens.txt
-rw-r--r--@   1 fangjun  staff   3.1M 12 May 14:53 voices.bin
build/bin/sherpa-onnx-offline-tts \
  --kitten-model=./kitten-micro-en-v0_8/model.onnx \
  --kitten-voices=./kitten-micro-en-v0_8/voices.bin \
  --kitten-tokens=./kitten-micro-en-v0_8/tokens.txt \
  --kitten-data-dir=./kitten-micro-en-v0_8/espeak-ng-data \
  --debug=1 \
  --sid=0 \
  --output-filename="./kitten-micro-0.wav" \
  "Today as always, men fall into two groups: slaves and free men. Whoever does not have two-thirds of his day for himself, is a slave, whatever he may be, a statesman, a businessman, an official, or a scholar."

Logs are:

/Users/fangjun/open-source/sherpa-onnx/sherpa-onnx/csrc/parse-options.cc:Read:373 build/bin/sherpa-onnx-offline-tts --kitten-model=./kitten-micro-en-v0_8/model.onnx --kitten-voices=./kitten-micro-en-v0_8/voices.bin --kitten-tokens=./kitten-micro-en-v0_8/tokens.txt --kitten-data-dir=./kitten-micro-en-v0_8/espeak-ng-data --debug=1 --sid=0 --output-filename=./kitten-micro-0.wav 'Today as always, men fall into two groups: slaves and free men. Whoever does not have two-thirds of his day for himself, is a slave, whatever he may be, a statesman, a businessman, an official, or a scholar.' 

OfflineTtsModelConfig(vits=OfflineTtsVitsModelConfig(model="", lexicon="", tokens="", data_dir="", noise_scale=0.667, noise_scale_w=0.8, length_scale=1), matcha=OfflineTtsMatchaModelConfig(acoustic_model="", vocoder="", lexicon="", tokens="", data_dir="", noise_scale=1, length_scale=1), kokoro=OfflineTtsKokoroModelConfig(model="", voices="", tokens="", lexicon="", data_dir="", length_scale=1, lang=""), zipvoice=OfflineTtsZipvoiceModelConfig(tokens="", encoder="", decoder="", vocoder="", data_dir="", lexicon="", feat_scale=0.1, t_shift=0.5, target_rms=0.1, guidance_scale=1), kitten=OfflineTtsKittenModelConfig(model="./kitten-micro-en-v0_8/model.onnx", voices="./kitten-micro-en-v0_8/voices.bin", tokens="./kitten-micro-en-v0_8/tokens.txt", data_dir="./kitten-micro-en-v0_8/espeak-ng-data", length_scale=1), pocket=OfflineTtsPocketModelConfig(lm_flow="", lm_main="", encoder="", decoder="", text_conditioner="", vocab_json="", token_scores_json="", voice_embedding_cache_capacity=50), supertonic=OfflineTtsSupertonicModelConfig(duration_predictor="", text_encoder="", vector_estimator="", vocoder="", tts_json="", unicode_indexer="", voice_style=""), num_threads=1, debug=True, provider="cpu")
/Users/fangjun/open-source/sherpa-onnx/sherpa-onnx/csrc/offline-tts-kitten-model.cc:Init:163 ---kitten model---
see_also=https://github.com/KittenML/KittenTTS
speaker_names=expr-voice-2-m,expr-voice-2-f,expr-voice-3-m,expr-voice-3-f,expr-voice-4-m,expr-voice-4-f,expr-voice-5-m,expr-voice-5-f
comment=This is KittenML/kitten-tts-micro-0.8 and supports only English
max_token_len=400
start_id=0
has_espeak=1
pad_id=0
voice=en-us
end_id=10
add_pad_after_end=1
model_url=https://huggingface.co/KittenML/kitten-tts-micro-0.8
language=English
id2speaker=0->expr-voice-2-m,1->expr-voice-2-f,2->expr-voice-3-m,3->expr-voice-3-f,4->expr-voice-4-m,5->expr-voice-4-f,6->expr-voice-5-m,7->expr-voice-5-f
version=8
n_speakers=8
maintainer=k2-fsa
sample_rate=24000
speaker2id=expr-voice-2-m->0,expr-voice-2-f->1,expr-voice-3-m->2,expr-voice-3-f->3,expr-voice-4-m->4,expr-voice-4-f->5,expr-voice-5-m->6,expr-voice-5-f->7
speaker_speed_priors=1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0
model_type=kitten-tts
style_dim=400,256
----------input names----------
0 input_ids
1 style
2 speed
----------output names----------
0 waveform
1 duration


/Users/fangjun/open-source/sherpa-onnx/sherpa-onnx/csrc/offline-tts-kitten-model.cc:Init:208 
0->expr-voice-2-m, 1->expr-voice-2-f, 2->expr-voice-3-m, 3->expr-voice-3-f, 4->expr-voice-4-m, 5->expr-voice-4-f, 6->expr-voice-5-m, 7->expr-voice-5-f, 


/Users/fangjun/open-source/sherpa-onnx/sherpa-onnx/csrc/offline-tts-kitten-impl.h:Generate:155 GenerationConfig(silence_scale=0.2, speed=1, sid=0, num_steps=5, reference_audio_len=0, reference_sample_rate=0)
/Users/fangjun/open-source/sherpa-onnx/sherpa-onnx/csrc/offline-tts-kitten-impl.h:Generate:202 Raw text: Today as always, men fall into two groups: slaves and free men. Whoever does not have two-thirds of his day for himself, is a slave, whatever he may be, a statesman, a businessman, an official, or a scholar.
/Users/fangjun/open-source/sherpa-onnx/sherpa-onnx/csrc/offline-tts-kitten-impl.h:Generate:216 In bytes (hex):
54 6f 64 61 79 20 61 73 20 61 6c 77 61 79 73 2c 20 6d 65 6e 20 66 61 6c 6c 20 69 6e 74 6f 20 74 77 6f 20 67 72 6f 75 70 73 3a 20 73 6c 61 76 65 73 20 61 6e 64 20 66 72 65 65 20 6d 65 6e 2e 20 57 68 6f 65 76 65 72 20 64 6f 65 73 20 6e 6f 74 20 68 61 76 65 20 74 77 6f 2d 74 68 69 72 64 73 20 6f 66 20 68 69 73 20 64 61 79 20 66 6f 72 20 68 69 6d 73 65 6c 66 2c 20 69 73 20 61 20 73 6c 61 76 65 2c 20 77 68 61 74 65 76 65 72 20 68 65 20 6d 61 79 20 62 65 2c 20 61 20 73 74 61 74 65 73 6d 61 6e 2c 20 61 20 62 75 73 69 6e 65 73 73 6d 61 6e 2c 20 61 6e 20 6f 66 66 69 63 69 61 6c 2c 20 6f 72 20 61 20 73 63 68 6f 6c 61 72 2e 

/Users/fangjun/open-source/sherpa-onnx/sherpa-onnx/csrc/offline-tts-kitten-impl.h:Generate:288 Split it into 2 batches. batch size: 1. Number of sentences: 2
Number of threads: 1
Elapsed seconds: 2.351 s
Audio duration: 13.481 s
Real-time factor (RTF): 2.351/13.481 = 0.174
The text is: Today as always, men fall into two groups: slaves and free men. Whoever does not have two-thirds of his day for himself, is a slave, whatever he may be, a statesman, a businessman, an official, or a scholar.. Speaker ID: 0
Saved to ./kitten-micro-0.wav successfully!
sample=107941, progress=0.500000
sample=215599, progress=1.000000

Generated audio:

kitten-micro-0.mov

Audio of different speakers (0-7)

sid 0

kitten-micro-0.mov

sid 1

kitten-micro-1.mov

sid 2

kitten-micro-2.mov

sid 3

kitten-micro-3.mov

sid 4

kitten-micro-4.mov

sid 5

kitten-micro-5.mov

sid 6

kitten-micro-6.mov

sid 7

kitten-micro-7.mov

Summary by CodeRabbit

  • New Features
    • Added support for KittenTTS offline text-to-speech model with local asset configuration.

Review Change Stack

@dosubot dosubot Bot added the size:M This PR changes 30-99 lines, ignoring generated files. label May 12, 2026
@coderabbitai

coderabbitai Bot commented May 12, 2026 •

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

This PR adds KittenTTS model support to the Node.js WASM runtime by introducing a new model type case for configuration, creating a build synchronization script to convert ES modules to CommonJS, and updating the CI/CD workflow to trigger on a different branch and publish to a new Hugging Face repository path.

Changes

KittenTTS WASM Integration and Export

Layer / File(s) Summary
Kitten Model Runtime Configuration
wasm/tts/sherpa-onnx-tts.node.js
Adds case 6 in the modelType switch to populate offlineTtsKittenModelConfig fields (model, voices, tokens, dataDir) with local asset paths for KittenTTS offline model support.
Node.js Build Synchronization Script
wasm/tts/sync-tts-node.py
Creates a new Python script that reads sherpa-onnx-tts.js, uses regex substitution to transform the ES module export into CommonJS module.exports format, and writes the output to sherpa-onnx-tts.node.js.
Export Workflow Configuration Updates
.github/workflows/export-kitten.yaml
Updates the workflow trigger from kitten-0.2 to update-kitten branch and changes Hugging Face repository URLs from csukuangfj to csukuangfj2 namespace for both git clone and push operations.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • k2-fsa/sherpa-onnx#3591: Both PRs modify KittenTTS-related WASM/Node TTS code and the export-kitten workflow for Kitten model support.
  • k2-fsa/sherpa-onnx#3263: Both PRs update GitHub Actions publishing workflows to change trigger branches and update the Hugging Face repository owner from csukuangfj to csukuangfj2.
  • k2-fsa/sherpa-onnx#2513: Both PRs modify .github/workflows/export-kitten.yaml and add/configure Kitten TTS model handling for workflow publishing.

Poem

🐰 A kitten now runs on the Node.js stage,
With config and sync scripts upon the page,
Module exports transform, ES to CommonJS flow,
The workflow branch updated—now publish, go!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Export KittenTTS models' is directly related to the main objective of the PR, which is to provide exported KittenTTS models. However, it is quite generic and doesn't convey specific details about what was actually changed in the code (workflow trigger updates, model configuration additions, Node.js export synchronization).
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds support for KittenTTS in the sherpa-onnx-tts.node.js file and introduces a Python script, sync-tts-node.py, to automate the conversion of ES modules to CommonJS. The review feedback suggests improving the Python script by explicitly specifying UTF-8 encoding for file operations and refactoring the regex replacement logic to enhance readability and ensure PEP 8 compliance.

Comment thread wasm/tts/sync-tts-node.py
src = os.path.join(dir_path, "sherpa-onnx-tts.js")
dst = os.path.join(dir_path, "sherpa-onnx-tts.node.js")

with open(src) as f:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

It is recommended to specify the encoding (e.g., encoding="utf-8") when opening files to ensure consistent behavior across different platforms and locales.

Suggested change
with open(src) as f:
with open(src, encoding="utf-8") as f:

Comment thread wasm/tts/sync-tts-node.py
content,
)

with open(dst, "w") as f:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

It is recommended to specify the encoding when opening the destination file for writing to maintain consistency.

Suggested change
with open(dst, "w") as f:
with open(dst, "w", encoding="utf-8") as f:

Comment thread wasm/tts/sync-tts-node.py
Comment on lines +18 to +22
content = re.sub(
r"export \{[^}]*\};",
"""if (typeof process == 'object' && typeof process.versions == 'object' &&\\n typeof process.versions.node == 'string') {\\n module.exports = {\\n createOfflineTts,\\n getDefaultOfflineTtsModelType,\\n };\\n}""",
content,
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The re.sub call uses an excessively long line with escaped newlines, which violates PEP 8 line length guidelines and reduces readability. Refactoring the replacement string into a separate variable using concatenated strings or a multi-line string is more idiomatic. Additionally, the regex can be made more robust by allowing for variable whitespace between the export keyword and the opening brace.

replacement = (\n    "if (typeof process == 'object' && typeof process.versions == 'object' &&\n"\n    "    typeof process.versions.node == 'string') {\n"\n    "  module.exports = {\n"\n    "    createOfflineTts,\n"\n    "    getDefaultOfflineTtsModelType,\n"\n    "  };\n"\n    "}"\n)\ncontent = re.sub(r"export\\s+\\{[^}]*\\};", replacement, content)

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
wasm/tts/sync-tts-node.py (1)

17-22: 💤 Low value

Consider validating the regex substitution.

The script performs a regex substitution but doesn't verify that it succeeded. If the export pattern changes in the source file, the script will silently produce incorrect output.

♻️ Optional: Add validation for the substitution
 # Replace ES module export with CommonJS module.exports
-content = re.sub(
+new_content = re.sub(
     r"export \{[^}]*\};",
     """if (typeof process == 'object' && typeof process.versions == 'object' &&\\n    typeof process.versions.node == 'string') {\\n  module.exports = {\\n    createOfflineTts,\\n    getDefaultOfflineTtsModelType,\\n  };\\n}""",
     content,
 )
+
+if new_content == content:
+    print("Warning: No export statement was replaced")
+
+content = new_content
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@wasm/tts/sync-tts-node.py` around lines 17 - 22, The regex substitution
replacing the ES export with a CommonJS module (using re.sub on variable
content) isn't validated and may silently fail; update the code to use re.subn
(or compare content before/after) when substituting the export pattern and check
the returned substitution count, and if it is zero raise/log an explicit error
(or exit) so you surface failures related to the export pattern not being found;
reference the existing regex, the re.sub/re.subn call, and the exported symbols
createOfflineTts and getDefaultOfflineTtsModelType when implementing the
validation.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@wasm/tts/sync-tts-node.py`:
- Around line 17-22: The regex substitution replacing the ES export with a
CommonJS module (using re.sub on variable content) isn't validated and may
silently fail; update the code to use re.subn (or compare content before/after)
when substituting the export pattern and check the returned substitution count,
and if it is zero raise/log an explicit error (or exit) so you surface failures
related to the export pattern not being found; reference the existing regex, the
re.sub/re.subn call, and the exported symbols createOfflineTts and
getDefaultOfflineTtsModelType when implementing the validation.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 4803c2d4-b338-4030-9baa-4576d9b11b62

📥 Commits

Reviewing files that changed from the base of the PR and between ceb1189 and ed2a913.

📒 Files selected for processing (3)
  • .github/workflows/export-kitten.yaml
  • wasm/tts/sherpa-onnx-tts.node.js
  • wasm/tts/sync-tts-node.py

@csukuangfj
csukuangfj merged commit 63d0d71 into k2-fsa:master May 12, 2026
1 check passed
@csukuangfj
csukuangfj deleted the update-kitten branch May 12, 2026 07:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant