Skip to content

feat(tts): pass lang_code to OpenAI-compatible TTS providers (Kokoro multilingual support) - #31693

Closed
ganzziani wants to merge 3 commits into
NousResearch:mainfrom
ganzziani:feat/tts-lang-code
Closed

ganzziani wants to merge 3 commits into
NousResearch:mainfrom
ganzziani:feat/tts-lang-code

Conversation

@ganzziani

Copy link
Copy Markdown
Contributor

Problem

When using Kokoro TTS (or any OpenAI-compatible TTS backend that supports lang_code) via the openai provider, the language field in config.yaml is ignored. The _generate_openai_tts() function reads model, voice, base_url, and speed but never passes language through to the API.

This means users configuring Kokoro for non-English languages (Spanish, French, etc.) get English phonemization — the voice reads the text with an English accent regardless of the language: es config setting.

Solution

Read the language config field from tts.openai.language and pass it as extra_body={"lang_code": language} to the OpenAI SDK's audio.speech.create() call. The OpenAI SDK passes extra_body fields directly to the underlying API, which Kokoro uses to select the correct phonemizer for the target language.

Changes

  • Add language = oai_config.get("language") to read the config value
  • Add extra_body={"lang_code": language} to create_kwargs when language is set
  • No breaking changes — language is optional and only used when explicitly configured

Config example

tts:
  provider: openai
  openai:
    model: kokoro
    voice: af_v0sarah
    language: es          # Now actually passed to Kokoro as lang_code=es
    base_url: http://localhost:8880/v1

Testing

Tested against Kokoro-FastAPI with Spanish text:

  • Without patch: af_v0sarah reads Spanish text with American English phonemization
  • With patch: same voice produces natural Spanish pronunciation via lang_code: es

…multilingual support)

The 'language' config field in tts.openai.language was read but never passed
to the API. This caused Kokoro (and other OpenAI-compatible TTS backends that
support lang_code) to default to English phonemization regardless of the
configured language.

Now passes lang_code via extra_body when language is set in config.
@alt-glitch alt-glitch added P3 Low — cosmetic, nice to have type/feature New feature or request tool/tts Text-to-speech and transcription labels May 24, 2026

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the focused OpenAI-compatible TTS improvement. The premise remains valid on current main: tools/tts_tool.py:1029-1068 reads OpenAI model, voice, base URL, and speed, then calls audio.speech.create() without forwarding a language value.

Problems

  • The PR adds request construction behavior without regression coverage. tests/tools/test_tts_speed.py:71-114 already captures audio.speech.create() kwargs, but has no lang_code case.
  • The setting is not documented in the OpenAI TTS config example at website/docs/user-guide/features/tts.md:54-58, which currently documents the compatible-endpoint base_url override.

Suggested changes

  • Add tests for configured language forwarding as extra_body={"lang_code": ...} and omission when unset.
  • Document tts.openai.language and that it is intended for OpenAI-compatible endpoints supporting lang_code.

This is an automated hermes-sweeper review.

Comment thread tools/tts_tool.py
}
if speed != 1.0:
create_kwargs["speed"] = max(0.25, min(4.0, speed))
if language:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a regression test for this branch using the existing mocked OpenAI TTS kwargs harness in tests/tools/test_tts_speed.py: assert extra_body == {"lang_code": "es"} when configured and that it is omitted when language is unset.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users labels Jul 13, 2026
Address review feedback on NousResearch#31693:
- Regression tests assert extra_body == {"lang_code": ...} is forwarded
  when tts.openai.language is configured, and omitted when unset/empty
- Document tts.openai.language as intended for OpenAI-compatible
  endpoints that support lang_code (e.g. Kokoro-FastAPI)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@ganzziani

Copy link
Copy Markdown
Contributor Author

Thanks for the review — both points are addressed in f26d93d:

Regression tests (tests/tools/test_tts_speed.py): new TestOpenaiTtsLangCode class covering:

  • tts.openai.language: "es" is forwarded as extra_body == {"lang_code": "es"}
  • extra_body is omitted when language is unset
  • extra_body is omitted when language is an empty string
  • a top-level tts.language is not forwarded (only the provider-scoped setting is honored)
  • lang_code and speed are forwarded independently in the same call

All 24 tests in the file pass locally.

Docs (website/docs/user-guide/features/tts.md): added language to the tts.openai config example and a note explaining it is intended for OpenAI-compatible endpoints that support lang_code (e.g. Kokoro-FastAPI), and should be left unset for the official OpenAI API.

Resolve docs conflict in tts.md (keep both the new Language section and
upstream Gemini Persona Prompts / Audio Tags sections) and update the
lang_code test mock for the new 3-tuple return of
_resolve_openai_audio_client_config.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
teknium1 pushed a commit that referenced this pull request Jul 28, 2026
Address review feedback on #31693:
- Regression tests assert extra_body == {"lang_code": ...} is forwarded
  when tts.openai.language is configured, and omitted when unset/empty
- Document tts.openai.language as intended for OpenAI-compatible
  endpoints that support lang_code (e.g. Kokoro-FastAPI)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@teknium1

Copy link
Copy Markdown
Collaborator

Merged into main via consolidated salvage PR #73513 (merge 4aac89b429). Your tts.openai.languagelang_code via extra_body (Kokoro multilingual) was cherry-picked (20a87e9 + fccf781) with your authorship, folded in as a config knob.

Your contribution is credited to you in git history. Thank you! Closing this PR as merged-via-salvage.

@teknium1 teknium1 closed this Jul 29, 2026
louisgreen0726 pushed a commit to louisgreen0726/hermes-agent that referenced this pull request Aug 7, 2026
Address review feedback on NousResearch#31693:
- Regression tests assert extra_body == {"lang_code": ...} is forwarded
  when tts.openai.language is configured, and omitted when unset/empty
- Document tts.openai.language as intended for OpenAI-compatible
  endpoints that support lang_code (e.g. Kokoro-FastAPI)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
Address review feedback on NousResearch#31693:
- Regression tests assert extra_body == {"lang_code": ...} is forwarded
  when tts.openai.language is configured, and omitted when unset/empty
- Document tts.openai.language as intended for OpenAI-compatible
  endpoints that support lang_code (e.g. Kokoro-FastAPI)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
prmartinow pushed a commit to prmartinow/hermes-agent that referenced this pull request Aug 26, 2026
Address review feedback on NousResearch#31693:
- Regression tests assert extra_body == {"lang_code": ...} is forwarded
  when tts.openai.language is configured, and omitted when unset/empty
- Document tts.openai.language as intended for OpenAI-compatible
  endpoints that support lang_code (e.g. Kokoro-FastAPI)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
melon-xf added a commit to melon-xf/hermes-agent that referenced this pull request Sep 3, 2026
Address review feedback on NousResearch#31693:
- Regression tests assert extra_body == {"lang_code": ...} is forwarded
  when tts.openai.language is configured, and omitted when unset/empty
- Document tts.openai.language as intended for OpenAI-compatible
  endpoints that support lang_code (e.g. Kokoro-FastAPI)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P3 Low — cosmetic, nice to have sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades tool/tts Text-to-speech and transcription type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants