Add emotion_id input to offline VITS TTS

[matiaslin]

Motivation

There are several TTS models that offer emotion selection. However, the current input shape for offline TTS is:

x, x_length, noise_scale, length_scale, noise_scale_w, sid

Note that there is no emotion input tensor parameter available. This forces the user who wants to use multi-emotion models, such as ai4bharat/vits_rasa_13, to export the model with a baked emotion (i.e. the emotion is immutable after the export).

So, to fully take advantage of the multi-emotion models, we would want to export it as intended: with the emotion being an input tensor of its own, namely emotion_id.

As of 6897144, if we export it with the emotion_id input tensor, and attempt to do TTS with sherpa-onnx-offline-tts, we would see the following error:

... [with T = onnxruntime::Tensor] Missing Input: emotion_id

This PR adds emotion_id as the optional 7th input tensor to offline VITS TTS to allow for emotion selection. This feature is only enabled when the model's metadata has num_emotions > 0 and the ONNX export has the emotion_id input tensor. That is to say that it's completely ignored, if either condition is not met.

Notes:

• Bindings: C++ API, C API
• New sherpa-onnx-offline-tts CLI flag: --emotion-id
• Resolves Emotions Support #2700.

Example Usage

Here is an example of how we vary the emotion using sherpa-onnx-vits-rasa-13 (my published ONNX export of ai4bharat/vits_rasa_13)

curl -SL -O https://huggingface.co/MatiasLin/sherpa-onnx-vits-rasa-13/resolve/main/model.onnx
curl -SL -O https://huggingface.co/MatiasLin/sherpa-onnx-vits-rasa-13/resolve/main/tokens.txt

sherpa-onnx-offline-tts \
  --vits-model=./model.onnx \
  --vits-tokens=./tokens.txt \
  --sid=0 \
  --emotion-id=0 \
  --output-filename=./rasa_emotion_id_0.wav \
  "வணககம, நஙகள எபபடி இருககிறரகள"

sherpa-onnx-offline-tts \
  --vits-model=./model.onnx \
  --vits-tokens=./tokens.txt \
  --sid=0 \
  --emotion-id=1 \
  --output-filename=./rasa_emotion_id_1.wav \
  "வணககம, நஙகள எபபடி இருககிறரகள"

Here is a sample output with only the emotion_id varying:

• rasa_emotion_id_0.wav
• rasa_emotion_id_1.wav

Summary by CodeRabbit

• New Features
  • Added emotion selection for multi-emotion VITS text-to-speech models.
  • Added an --emotion-id command-line option.
  • Emotion settings now work with both single and batched synthesis.
• Bug Fixes
  • Unsupported or out-of-range emotion IDs safely fall back to the default emotion with diagnostic logging.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 99ee60d3-e366-4039-9d7e-793e0855f3e9

📥 Commits

Reviewing files that changed from the base of the PR and between 39c45ab and 3289bb5.

📒 Files selected for processing (1)

• sherpa-onnx/csrc/sherpa-onnx-offline-tts.cc

🚧 Files skipped from review as they are similar to previous changes (1)

• sherpa-onnx/csrc/sherpa-onnx-offline-tts.cc

---

📝 Walkthrough

Walkthrough

VITS TTS now accepts emotion_id from the CLI and GenerationConfig. The implementation validates the value, propagates it through synthesis, and supplies emotion tensors to compatible models.

Changes

VITS emotion support

Layer / File(s)	Summary
Generation configuration and API wiring sherpa-onnx/csrc/sherpa-onnx-offline-tts.cc	The CLI registers --emotion-id and stores nonnegative values in GenerationConfig.
Generation validation and propagation sherpa-onnx/csrc/offline-tts-vits-impl.h	VITS generation reads and validates emotion_id, resets unsupported or out-of-range values to zero, and passes the value through all synthesis paths.
VITS model emotion input sherpa-onnx/csrc/offline-tts-vits-model-meta-data.h, sherpa-onnx/csrc/offline-tts-vits-model.h, sherpa-onnx/csrc/offline-tts-vits-model.cc	Model metadata records num_emotions. Run accepts emotion_id and supplies speaker and emotion tensors according to the model inputs.
VITS API documentation sherpa-onnx/csrc/offline-tts-vits-model.h, sherpa-onnx/csrc/offline-tts-vits-impl.h	The VITS APIs document emotion IDs and the required model metadata and tensor inputs.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant TTSCLI
  participant OfflineTtsVitsImpl
  participant OfflineTtsVitsModel
  TTSCLI->>OfflineTtsVitsImpl: provide emotion_id through GenerationConfig
  OfflineTtsVitsImpl->>OfflineTtsVitsImpl: validate emotion_id
  OfflineTtsVitsImpl->>OfflineTtsVitsModel: Run with emotion_id
  OfflineTtsVitsModel->>OfflineTtsVitsModel: create emotion tensor when supported

Loading

Possibly related PRs

• k2-fsa/sherpa-onnx#3546: Both changes modify VITS model metadata and inference-related code, but this PR adds emotion-ID support.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 5.88% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the main change: adding emotion_id input support to offline VITS TTS.
Linked Issues check	✅ Passed	The changes implement emotion selection for offline VITS TTS, which directly addresses the emotion support requested in issue #2700.
Out of Scope Changes check	✅ Passed	All summarized changes support emotion_id handling, validation, propagation, model execution, metadata, or CLI configuration for offline VITS TTS.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[csukuangfj]

Pleasr add it to the generation config extra field. The change is very small.and you don't need to change so many files.

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/dotnet/OfflineTtsGenerationConfig.cs`:
- Line 138: Add a managed EmotionId field to OfflineTtsGenerationConfig and
update its ToNative conversion to assign that value to NativeStruct.EmotionId,
preserving the existing configuration mapping so .NET callers can select
non-zero emotions.

In `@wasm/tts/sherpa-onnx-tts.js`:
- Around line 762-763: Update generate’s genConfig construction to include
config.emotionId, defaulting with nullish semantics to 0 before passing it to
initSherpaOnnxGenerationConfig, so generate({text, emotionId}) propagates the
requested emotion ID.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: e633017b-b50e-4467-8282-445229f0c3a5

📥 Commits

Reviewing files that changed from the base of the PR and between 9860cb0 and a79b056.

📒 Files selected for processing (8)

• flutter/sherpa_onnx/lib/src/sherpa_onnx_bindings.dart
• scripts/dotnet/OfflineTtsGenerationConfig.cs
• sherpa-onnx/pascal-api/sherpa_onnx.pas
• sherpa-onnx/rust/sherpa-onnx-sys/src/tts.rs
• sherpa-onnx/rust/sherpa-onnx/src/tts.rs
• swift-api-examples/SherpaOnnx.swift
• wasm/tts/sherpa-onnx-tts.js
• wasm/tts/sherpa-onnx-wasm-main-tts.cc

[matiaslin]

Pleasr add it to the generation config extra field. The change is very small.and you don't need to change so many files.

Hmm, I see what you mean. Yeah, we can mimick what we already do with "lang". Ok, I'll push another patch soon.

[matiaslin]

Done. Added emotion_id as an extra param in GenerationConfig.

[matiaslin]

The failing checks are unrelated to this change. Ready for review. Thank you.

[csukuangfj]

Thank you for your contribution!

[WilliamTambellini]

Tks @csukuangfj