Repository navigation
Add C API for KittenTTS - #2467
Conversation
|
Note Other AI code review bot(s) detectedCodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review. Caution Review failedThe pull request is closed. WalkthroughA new example for English text-to-speech (TTS) using the Kitten model was introduced to the C API, including a demonstration C file, CMake integration, and workflow automation for building and testing. The C API and its configuration structures were updated to support Kitten TTS models, with corresponding internal logic and public struct changes. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant KittenTTS_C_Example
participant SherpaOnnxCAPI
participant KittenModelFiles
User->>KittenTTS_C_Example: Run main()
KittenTTS_C_Example->>KittenModelFiles: Load model, voices, tokens
KittenTTS_C_Example->>SherpaOnnxCAPI: Initialize TTS config (Kitten)
KittenTTS_C_Example->>SherpaOnnxCAPI: Synthesize text to audio
SherpaOnnxCAPI-->>KittenTTS_C_Example: Return generated audio
KittenTTS_C_Example->>User: Write WAV file, print info
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~15 minutes Possibly related PRs
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (6)
✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Pull Request Overview
This pull request adds C API support for KittenTTS, a text-to-speech engine, to the sherpa-onnx library. The changes enable users to use KittenTTS models through the existing C API interface.
- Adds
SherpaOnnxOfflineTtsKittenModelConfigstruct to configure KittenTTS models - Integrates Kitten configuration into the existing TTS model configuration structure
- Provides a complete example demonstrating KittenTTS usage with English text-to-speech
Reviewed Changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| sherpa-onnx/c-api/c-api.h | Defines the KittenTTS configuration structure and integrates it into the main TTS model config |
| sherpa-onnx/c-api/c-api.cc | Implements configuration mapping from C API structs to internal C++ configuration |
| c-api-examples/kitten-tts-en-c-api.c | Provides a complete example showing how to use KittenTTS for English text-to-speech |
| c-api-examples/CMakeLists.txt | Adds build configuration for the KittenTTS example |
| .github/workflows/c-api.yaml | Adds CI testing for the KittenTTS functionality |
| tts_config.model.kitten.data_dir = | ||
| SHERPA_ONNX_OR(config->model.kitten.data_dir, ""); | ||
| tts_config.model.kitten.length_scale = | ||
| SHERPA_ONNX_OR(config->model.kitten.length_scale, 1.0); |
There was a problem hiding this comment.
The SHERPA_ONNX_OR macro is being used with a float value (1.0) as the default, but this macro appears to be designed for pointer/string checks based on its usage with string fields above. This may not work correctly for float values and could cause compilation errors or unexpected behavior.
| SHERPA_ONNX_OR(config->model.kitten.length_scale, 1.0); | |
| (config->model.kitten.length_scale != 0.0 ? config->model.kitten.length_scale : 1.0); |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
c-api-examples/kitten-tts-en-c-api.c (1)
61-62: Clarify speed vs length_scale semanticsHeader docs say smaller length_scale => faster. The example’s comment says larger speed => faster. If speed is internally mapped as 1/length_scale, consider adding a short note to prevent confusion (done in the diff above).
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (5)
.github/workflows/c-api.yaml(1 hunks)c-api-examples/CMakeLists.txt(1 hunks)c-api-examples/kitten-tts-en-c-api.c(1 hunks)sherpa-onnx/c-api/c-api.cc(1 hunks)sherpa-onnx/c-api/c-api.h(1 hunks)
🧰 Additional context used
🧠 Learnings (2)
📚 Learning: 2025-08-06T04:23:50.237Z
Learnt from: litongjava
PR: k2-fsa/sherpa-onnx#2440
File: sherpa-onnx/java-api/src/main/java/com/k2fsa/sherpa/onnx/core/Core.java:4-6
Timestamp: 2025-08-06T04:23:50.237Z
Learning: The sherpa-onnx JNI library files are stored in Hugging Face repository at https://huggingface.co/csukuangfj/sherpa-onnx-libs under versioned directories like jni/1.12.7/, and the actual Windows JNI library filename is "sherpa-onnx-jni.dll" as defined in Core.java constants.
Applied to files:
sherpa-onnx/c-api/c-api.hc-api-examples/CMakeLists.txt
📚 Learning: 2025-08-06T04:18:47.981Z
Learnt from: litongjava
PR: k2-fsa/sherpa-onnx#2440
File: sherpa-onnx/java-api/src/main/java/com/k2fsa/sherpa/onnx/core/Core.java:4-6
Timestamp: 2025-08-06T04:18:47.981Z
Learning: In sherpa-onnx Java API, the native library names in Core.java (WIN_NATIVE_LIBRARY_NAME = "sherpa-onnx-jni.dll", UNIX_NATIVE_LIBRARY_NAME = "libsherpa-onnx-jni.so", MACOS_NATIVE_LIBRARY_NAME = "libsherpa-onnx-jni.dylib") are copied directly from the compiled binary filenames and should not be changed to match other libraries' naming conventions.
Applied to files:
sherpa-onnx/c-api/c-api.hc-api-examples/CMakeLists.txt
🔇 Additional comments (5)
c-api-examples/CMakeLists.txt (2)
26-28: LGTM: New Kitten TTS example target is correctly gated and linked
- Target is added under SHERPA_ONNX_ENABLE_TTS, consistent with other TTS examples.
- Linking against sherpa-onnx-c-api matches matcha/kokoro examples (no cargs unless the example uses it).
26-28: Verification complete: example file and dependencies are correctly configuredAll checks passed:
- c-api-examples/kitten-tts-en-c-api.c exists.
- .github/workflows/c-api.yaml references
kitten-tts-en-c-api.- No CLI parsing in the example; linking without cargs is appropriate.
No changes required.
sherpa-onnx/c-api/c-api.cc (2)
1210-1221: LGTM: Kitten TTS config mapped with sane defaults and parity with other models
- Fields (model, voices, tokens, data_dir, length_scale) are wired via SHERPA_ONNX_OR with consistent defaults (length_scale=1.0).
- Placement aligns with existing blocks (vits/matcha/kokoro). No obvious regressions.
1210-1221: Kitten TTS integration points are properly wired upAll requested checks pass without issue:
- The C-API header (
c-api/c-api.h) definesSherpaOnnxOfflineTtsKittenModelConfigand embeds it as thekittenfield inSherpaOnnxOfflineTtsModelConfig.- The offline-TTS implementation (
csrc/offline-tts-impl.cc) includesoffline-tts-kitten-impl.hand branches onconfig.model.kitten.model.OfflineTtsKittenModelConfig::Validate()andToString()are called incsrc/offline-tts-model-config.cc(lines 46 and 61).OfflineTtsConfig::Validate()andToString()incsrc/offline-tts.cchandle the kitten model.- The loader and runtime for
model.kittenare implemented inoffline-tts-kitten-model.cc/.h.No further action required.
sherpa-onnx/c-api/c-api.h (1)
1051-1051: Struct growth and ABI considerationsAppending
kittenat the end ofSherpaOnnxOfflineTtsModelConfigis the least risky option for source compatibility. For runtime ABI compatibility, ensure clients recompile against this header before using a newer shared library; otherwise, the callee may read beyond the caller-allocated struct.
- If this is intended as a minor version bump, please reflect it in release notes. The workflow already runs
./new-release.sh.
| - name: Test KittenTTS | ||
| shell: bash | ||
| run: | | ||
| name=kitten-tts-en-c-api | ||
| gcc -o $name ./c-api-examples/$name.c \ | ||
| -I ./build/install/include \ | ||
| -L ./build/install/lib/ \ | ||
| -l sherpa-onnx-c-api \ | ||
| -l onnxruntime | ||
|
|
||
| ls -lh $name | ||
|
|
||
| if [[ ${{ matrix.os }} == ubuntu-latest || ${{ matrix.os }} == ubuntu-22.04-arm ]]; then | ||
| ldd ./$name | ||
| echo "----" | ||
| readelf -d ./$name | ||
| fi | ||
|
|
||
| curl -SL -O https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/kitten-nano-en-v0_1-fp16.tar.bz2 | ||
| tar xf kitten-nano-en-v0_1-fp16.tar.bz2 | ||
| rm kitten-nano-en-v0_1-fp16.tar.bz2 | ||
|
|
||
| export LD_LIBRARY_PATH=$PWD/build/install/lib:$LD_LIBRARY_PATH | ||
| export DYLD_LIBRARY_PATH=$PWD/build/install/lib:$DYLD_LIBRARY_PATH | ||
|
|
||
| ./$name | ||
|
|
||
| rm $name | ||
| rm -rf kitten-nano-en-v0_1-fp16 | ||
|
|
||
| - uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: kitten-tts-wavs-${{ matrix.os }} | ||
| path: ./generated-kitten-en.wav | ||
|
|
There was a problem hiding this comment.
💡 Verification agent
❓ Verification inconclusive
Harden the KittenTTS CI step (fail fast, robust downloads, rpath, artifact existence)
- Add
set -euo pipefailso failures propagate. - Use
curl -fSLto fail on HTTP errors. - Optionally embed rpath so LD/DYLD_LIBRARY_PATH isn’t required.
- Verify the WAV exists before uploading.
Apply this diff within the step:
- name: Test KittenTTS
shell: bash
run: |
- name=kitten-tts-en-c-api
+ set -euo pipefail
+ name=kitten-tts-en-c-api
gcc -o $name ./c-api-examples/$name.c \
-I ./build/install/include \
-L ./build/install/lib/ \
- -l sherpa-onnx-c-api \
- -l onnxruntime
+ -l sherpa-onnx-c-api \
+ -l onnxruntime \
+ -Wl,-rpath,'$ORIGIN/build/install/lib'
ls -lh $name
if [[ ${{ matrix.os }} == ubuntu-latest || ${{ matrix.os }} == ubuntu-22.04-arm ]]; then
ldd ./$name
echo "----"
readelf -d ./$name
fi
- curl -SL -O https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/kitten-nano-en-v0_1-fp16.tar.bz2
+ curl -fSL -O https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/kitten-nano-en-v0_1-fp16.tar.bz2
tar xf kitten-nano-en-v0_1-fp16.tar.bz2
rm kitten-nano-en-v0_1-fp16.tar.bz2
export LD_LIBRARY_PATH=$PWD/build/install/lib:$LD_LIBRARY_PATH
export DYLD_LIBRARY_PATH=$PWD/build/install/lib:$DYLD_LIBRARY_PATH
./$name
+ test -f generated-kitten-en.wav
rm $name
rm -rf kitten-nano-en-v0_1-fp16Note: If macOS fails due to $ORIGIN, replace it with @loader_path in the rpath flag for Darwin.
Harden the “Test KittenTTS” Bash step
Introduce early‐exit, strict shell flags, robust download checks, embedded rpath, and artifact validation:
- Prepend
set -euo pipefailat the top of therun:block to fail fast on any error or unset variable. - Switch
curl -SLtocurl -fSLso HTTP errors abort the script. - Add
-Wl,-rpath,'$ORIGIN/build/install/lib'to thegcccommand line (use@loader_pathon macOS) to avoid relying onLD_LIBRARY_PATH/DYLD_LIBRARY_PATH. - After running the binary, verify the WAV was actually created with
test -f generated-kitten-en.wavbefore uploading.
- name: Test KittenTTS
shell: bash
run: |
+ set -euo pipefail
name=kitten-tts-en-c-api
gcc -o $name ./c-api-examples/$name.c \
-I ./build/install/include \
-L ./build/install/lib/ \
- -l sherpa-onnx-c-api \
- -l onnxruntime
+ -l sherpa-onnx-c-api \
+ -l onnxruntime \
+ -Wl,-rpath,'$ORIGIN/build/install/lib'
ls -lh $name
if [[ ${{ matrix.os }} == ubuntu-latest || ${{ matrix.os }} == ubuntu-22.04-arm ]]; then
ldd ./$name
echo "----"
readelf -d ./$name
fi
- curl -SL -O https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/kitten-nano-en-v0_1-fp16.tar.bz2
+ curl -fSL -O https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/kitten-nano-en-v0_1-fp16.tar.bz2
tar xf kitten-nano-en-v0_1-fp16.tar.bz2
rm kitten-nano-en-v0_1-fp16.tar.bz2
export LD_LIBRARY_PATH=$PWD/build/install/lib:$LD_LIBRARY_PATH
export DYLD_LIBRARY_PATH=$PWD/build/install/lib:$DYLD_LIBRARY_PATH
./$name
+ test -f generated-kitten-en.wav
rm $name
rm -rf kitten-nano-en-v0_1-fp16Note: On Darwin replace $ORIGIN with @loader_path in the rpath flag.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| - name: Test KittenTTS | |
| shell: bash | |
| run: | | |
| name=kitten-tts-en-c-api | |
| gcc -o $name ./c-api-examples/$name.c \ | |
| -I ./build/install/include \ | |
| -L ./build/install/lib/ \ | |
| -l sherpa-onnx-c-api \ | |
| -l onnxruntime | |
| ls -lh $name | |
| if [[ ${{ matrix.os }} == ubuntu-latest || ${{ matrix.os }} == ubuntu-22.04-arm ]]; then | |
| ldd ./$name | |
| echo "----" | |
| readelf -d ./$name | |
| fi | |
| curl -SL -O https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/kitten-nano-en-v0_1-fp16.tar.bz2 | |
| tar xf kitten-nano-en-v0_1-fp16.tar.bz2 | |
| rm kitten-nano-en-v0_1-fp16.tar.bz2 | |
| export LD_LIBRARY_PATH=$PWD/build/install/lib:$LD_LIBRARY_PATH | |
| export DYLD_LIBRARY_PATH=$PWD/build/install/lib:$DYLD_LIBRARY_PATH | |
| ./$name | |
| rm $name | |
| rm -rf kitten-nano-en-v0_1-fp16 | |
| - uses: actions/upload-artifact@v4 | |
| with: | |
| name: kitten-tts-wavs-${{ matrix.os }} | |
| path: ./generated-kitten-en.wav | |
| - name: Test KittenTTS | |
| shell: bash | |
| run: | | |
| set -euo pipefail | |
| name=kitten-tts-en-c-api | |
| gcc -o $name ./c-api-examples/$name.c \ | |
| -I ./build/install/include \ | |
| -L ./build/install/lib/ \ | |
| -l sherpa-onnx-c-api \ | |
| -l onnxruntime \ | |
| -Wl,-rpath,'$ORIGIN/build/install/lib' | |
| ls -lh $name | |
| if [[ ${{ matrix.os }} == ubuntu-latest || ${{ matrix.os }} == ubuntu-22.04-arm ]]; then | |
| ldd ./$name | |
| echo "----" | |
| readelf -d ./$name | |
| fi | |
| curl -fSL -O https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/kitten-nano-en-v0_1-fp16.tar.bz2 | |
| tar xf kitten-nano-en-v0_1-fp16.tar.bz2 | |
| rm kitten-nano-en-v0_1-fp16.tar.bz2 | |
| export LD_LIBRARY_PATH=$PWD/build/install/lib:$LD_LIBRARY_PATH | |
| export DYLD_LIBRARY_PATH=$PWD/build/install/lib:$DYLD_LIBRARY_PATH | |
| ./$name | |
| test -f generated-kitten-en.wav | |
| rm $name | |
| rm -rf kitten-nano-en-v0_1-fp16 | |
| - uses: actions/upload-artifact@v4 | |
| with: | |
| name: kitten-tts-wavs-${{ matrix.os }} | |
| path: ./generated-kitten-en.wav |
🤖 Prompt for AI Agents
In .github/workflows/c-api.yaml between lines 88 and 122, enhance the "Test
KittenTTS" bash step by adding `set -euo pipefail` at the start of the run block
to enable strict error handling and fail fast on unset variables. Modify the
curl command to use `curl -fSL` to ensure HTTP errors cause the script to abort.
Update the gcc command to include the rpath linker flag: use
`-Wl,-rpath,'$ORIGIN/build/install/lib'` on Linux and
`-Wl,-rpath,@loader_path/build/install/lib` on macOS to embed the library path
and avoid relying on environment variables. Finally, after executing the binary,
add a check with `test -f generated-kitten-en.wav` to confirm the WAV file was
created before uploading the artifact.
| SHERPA_ONNX_API typedef struct SherpaOnnxOfflineTtsKittenModelConfig { | ||
| const char *model; | ||
| const char *voices; | ||
| const char *tokens; | ||
| const char *data_dir; | ||
|
|
||
| float length_scale; // < 1, faster in speech speed; > 1, slower in speed | ||
| } SherpaOnnxOfflineTtsKittenModelConfig; | ||
|
|
There was a problem hiding this comment.
💡 Verification agent
🧩 Analysis chain
Document required/optional fields and clarify defaults for Kitten config
- Please add brief docs mirroring other TTS structs: which files are required (model/voices/tokens/data_dir) and whether length_scale has a default (e.g., 1.0 when 0 or unset).
- The example leaves length_scale unset (0 after memset). If c-api.cc defaults it to 1.0, note that here to avoid confusion.
To confirm default handling in the implementation:
🏁 Script executed:
#!/bin/bash
# Verify kitten config handling and defaults in the implementation
rg -n "GetOfflineTtsConfig|Kitten|kitten" -A 10 -B 10 sherpa-onnx | sed -n '1,200p'Length of output: 14951
🏁 Script executed:
#!/bin/bash
# Locate and inspect the C++ core kitten model config implementation
echo "C++ core kitten model config files:"
fd -HI -g 'offline-tts-kitten-model-config.*'
for f in $(fd -HI -g 'offline-tts-kitten-model-config.*'); do
echo "=== $f ==="
sed -n '1,100p' "$f"
doneLength of output: 6864
🏁 Script executed:
#!/bin/bash
# Inspect C API conversion for Kitten TTS config defaults
echo "Searching for C API struct definition and conversion usage..."
rg -n "SherpaOnnxOfflineTtsKittenModelConfig" -A5 -B5 sherpa-onnx/c-api
echo ""
echo "Searching where Kitten config is mapped to C++ core struct..."
rg -n "kitten" -A10 -B5 sherpa-onnx/c-apiLength of output: 5030
Add documentation for Kitten TTS config fields and clarify defaults
Please update sherpa-onnx/c-api/c-api.h to mirror the style of the existing TTS structs (VITS, Matcha, Kokoro) by annotating which fields are required and what defaults apply:
• In typedef struct SherpaOnnxOfflineTtsKittenModelConfig { … }:
– const char *model;
‣ Path to the kitten ONNX model. Required.
– const char *voices;
‣ Path to voices.bin for kitten. Required.
– const char *tokens;
‣ Path to tokens.txt for kitten. Required.
– const char *data_dir;
‣ Directory containing espeak-ng files (phontab, phonindex, etc.). Required.
– float length_scale; // <1 faster; >1 slower
‣ Inverse of speech speed. Optional. Defaults to 1.0 if zero or unset (as enforced by the C++ core and in the C API mapping via SHERPA_ONNX_OR(..., 1.0)).
• In SherpaOnnxOfflineTtsModelConfig, you can leave the kitten member where it is (added at the end), but ensure consumers know this is a new field and may affect ABI compatibility.
Refer to the comment blocks on SherpaOnnxOfflineTtsOfflineTtsMatchaModelConfig and SherpaOnnxOfflineTtsKokoroModelConfig for examples of how required/optional fields and defaults are documented.
🤖 Prompt for AI Agents
In sherpa-onnx/c-api/c-api.h around lines 1035 to 1043, add detailed comments to
each field of SherpaOnnxOfflineTtsKittenModelConfig to specify which are
required and which are optional with defaults, following the style used in
SherpaOnnxOfflineTtsMatchaModelConfig and SherpaOnnxOfflineTtsKokoroModelConfig.
Document that model, voices, tokens, and data_dir are required paths, and
length_scale is optional with a default of 1.0 if unset or zero. Also, in
SherpaOnnxOfflineTtsModelConfig, keep the kitten member at the end but add a
note that this is a new field that may impact ABI compatibility.
Summary by CodeRabbit