Skip to content

Add CXX API for supertonic TTS - #3279

Closed
Wasser1462 wants to merge 24 commits into
k2-fsa:masterfrom
Wasser1462:feature/supertonic-cxx-api
Closed

Wasser1462 wants to merge 24 commits into
k2-fsa:masterfrom
Wasser1462:feature/supertonic-cxx-api

Conversation

@Wasser1462

@Wasser1462 Wasser1462 commented Mar 10, 2026 •

Copy link
Copy Markdown
Collaborator

Summary by CodeRabbit

  • New Features
    • Extended text-to-speech capabilities with support for the Supertonic TTS model, providing an additional model option for speech synthesis.
    • Added a new example demonstrating English speech synthesis using Supertonic with configurable generation parameters and progress tracking.

@dosubot dosubot Bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Mar 10, 2026
@coderabbitai

coderabbitai Bot commented Mar 10, 2026 •

Copy link
Copy Markdown

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

This PR extends the sherpa-onnx C/C++ API to support the Supertonic TTS model by introducing new configuration structures, populating these fields through the API layers, adding a CMake build target, and providing a complete C++ example demonstrating Supertonic English TTS synthesis with progress callbacks.

Changes

Cohort / File(s) Summary
Supertonic Model Configuration (Headers)
sherpa-onnx/c-api/c-api.h, sherpa-onnx/c-api/cxx-api.h
Added new SherpaOnnxOfflineTtsSupertonicModelConfig and OfflineTtsSupertonicModelConfig structs with seven model component paths (duration_predictor, text_encoder, vector_estimator, vocoder, tts_json, unicode_indexer, voice_style), extended respective OfflineTtsModelConfig structs with supertonic field.
API Implementation
sherpa-onnx/c-api/c-api.cc, sherpa-onnx/c-api/cxx-api.cc
Populated new Supertonic model fields from input configuration in GetOfflineTtsConfig and OfflineTts::Create; enhanced JSON value handling in SherpaOnnxOfflineTtsGenerateInternal to conditionally convert non-string values using dump() vs. direct string assignment.
Build Configuration
cxx-api-examples/CMakeLists.txt
Added supertonic-tts-en-cxx-api executable target with dependency linking to sherpa-onnx-cxx-api, mirroring existing TTS example structure.
Supertonic TTS Example
cxx-api-examples/supertonic-tts-en-cxx-api.cc
New C++ example demonstrating Supertonic English TTS usage: configures OfflineTts with model paths and parameters, defines ProgressCallback for generation progress reporting, synthesizes text to audio via GenerationConfig, and persists output to WAV file.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related issues

Possibly related PRs

Suggested labels

size:M

Suggested reviewers

  • csukuangfj

Poem

🐰 A new voice joins the chorus bright,
Supertonic sings with ONNX might,
Through C and C++, the configs flow,
Duration, text, and vocoders glow—
Another TTS example hops in delight!

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main objective of the pull request, which adds C++ API support for Supertonic TTS across multiple files and configuration structures.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Tip

Try Coding Plans. Let us write the prompt for your AI agent so you can ship faster (with fewer bugs).
Share your feedback on Discord.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@Wasser1462 Wasser1462 closed this Mar 10, 2026
@gemini-code-assist

Copy link
Copy Markdown

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request integrates the Supertonic text-to-speech model into the existing framework, enhancing the system's speech synthesis capabilities. It introduces the necessary configuration structures and API mappings to support the new model, alongside a practical C++ example to demonstrate its usage and features like progress callbacks and generation parameter adjustments.

Highlights

  • New TTS Model Support: Extended text-to-speech capabilities by adding support for the Supertonic TTS model, providing an additional option for speech synthesis.
  • New C++ Example: Added a new C++ example demonstrating English speech synthesis using the Supertonic model, including configurable generation parameters and progress tracking.
Changelog
  • cxx-api-examples/CMakeLists.txt
    • Added a new executable target for the Supertonic TTS English CXX API example.
  • cxx-api-examples/supertonic-tts-en-cxx-api.cc
    • Introduced a new C++ example for English text-to-speech using the Supertonic model, including configurable generation parameters and progress tracking.
  • sherpa-onnx/c-api/c-api.cc
    • Configured the Supertonic model parameters within the C API's OfflineTtsConfig initialization.
    • Improved the JSON parsing logic for extra generation configuration to correctly handle string values, preventing incorrect serialization.
  • sherpa-onnx/c-api/c-api.h
    • Defined a new C API struct SherpaOnnxOfflineTtsSupertonicModelConfig to encapsulate Supertonic TTS model parameters.
    • Integrated the SherpaOnnxOfflineTtsSupertonicModelConfig into the SherpaOnnxOfflineTtsModelConfig struct.
  • sherpa-onnx/c-api/cxx-api.cc
    • Mapped Supertonic model configuration parameters from the C++ API's OfflineTtsConfig to the underlying C API structure during OfflineTts::Create.
  • sherpa-onnx/c-api/cxx-api.h
    • Defined a new C++ API struct OfflineTtsSupertonicModelConfig to hold Supertonic TTS model parameters.
    • Integrated the OfflineTtsSupertonicModelConfig into the OfflineTtsModelConfig C++ struct.
Activity
  • The pull request description includes an auto-generated summary by CodeRabbit, detailing new features related to Supertonic TTS support and an example.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds CXX API support for the Supertonic TTS model. The changes include adding new configuration structures for Supertonic in both the C and CXX APIs, implementing the logic to handle these configurations, and providing a new example file to demonstrate its usage. The implementation looks solid and consistent with the existing structure for other TTS models. I've included one suggestion to improve the maintainability of the new example file by refactoring hardcoded model paths.

Comment on lines +39 to +53
config.model.supertonic.duration_predictor =
"./sherpa-onnx-supertonic-tts-int8-2026-03-06/"
"duration_predictor.int8.onnx";
config.model.supertonic.text_encoder =
"./sherpa-onnx-supertonic-tts-int8-2026-03-06/text_encoder.int8.onnx";
config.model.supertonic.vector_estimator =
"./sherpa-onnx-supertonic-tts-int8-2026-03-06/vector_estimator.int8.onnx";
config.model.supertonic.vocoder =
"./sherpa-onnx-supertonic-tts-int8-2026-03-06/vocoder.int8.onnx";
config.model.supertonic.tts_json =
"./sherpa-onnx-supertonic-tts-int8-2026-03-06/tts.json";
config.model.supertonic.unicode_indexer =
"./sherpa-onnx-supertonic-tts-int8-2026-03-06/unicode_indexer.bin";
config.model.supertonic.voice_style =
"./sherpa-onnx-supertonic-tts-int8-2026-03-06/voice.bin";

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To improve maintainability and reduce redundancy, consider defining the base path for the model files in a constant string and then constructing the full paths from it. This makes it easier to update the model path in the future.

Suggested change
config.model.supertonic.duration_predictor =
"./sherpa-onnx-supertonic-tts-int8-2026-03-06/"
"duration_predictor.int8.onnx";
config.model.supertonic.text_encoder =
"./sherpa-onnx-supertonic-tts-int8-2026-03-06/text_encoder.int8.onnx";
config.model.supertonic.vector_estimator =
"./sherpa-onnx-supertonic-tts-int8-2026-03-06/vector_estimator.int8.onnx";
config.model.supertonic.vocoder =
"./sherpa-onnx-supertonic-tts-int8-2026-03-06/vocoder.int8.onnx";
config.model.supertonic.tts_json =
"./sherpa-onnx-supertonic-tts-int8-2026-03-06/tts.json";
config.model.supertonic.unicode_indexer =
"./sherpa-onnx-supertonic-tts-int8-2026-03-06/unicode_indexer.bin";
config.model.supertonic.voice_style =
"./sherpa-onnx-supertonic-tts-int8-2026-03-06/voice.bin";
const std::string model_dir = "./sherpa-onnx-supertonic-tts-int8-2026-03-06/";
config.model.supertonic.duration_predictor =
model_dir + "duration_predictor.int8.onnx";
config.model.supertonic.text_encoder = model_dir + "text_encoder.int8.onnx";
config.model.supertonic.vector_estimator =
model_dir + "vector_estimator.int8.onnx";
config.model.supertonic.vocoder = model_dir + "vocoder.int8.onnx";
config.model.supertonic.tts_json = model_dir + "tts.json";
config.model.supertonic.unicode_indexer =
model_dir + "unicode_indexer.bin";
config.model.supertonic.voice_style = model_dir + "voice.bin";

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant