Skip to content

feat(stt): add Yandex SpeechKit STT provider - #23995

Closed
ibaldr89 wants to merge 1 commit into
NousResearch:mainfrom
ibaldr89:feat/yandex-stt-provider
Closed

feat(stt): add Yandex SpeechKit STT provider#23995
ibaldr89 wants to merge 1 commit into
NousResearch:mainfrom
ibaldr89:feat/yandex-stt-provider

Conversation

@ibaldr89

@ibaldr89 ibaldr89 commented May 11, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Adds Yandex SpeechKit Sync API as a seventh STT provider. Yandex SpeechKit
has best-in-class Russian speech recognition accuracy (~97% WER). It natively
accepts OGG/Opus, which is the default voice format sent by Telegram and
WhatsApp — meaning no re-encoding is needed for the most common gateway use case.

The provider slots into the existing STT abstraction with zero changes to
callers. It lives entirely within tools/transcription_tools.py.

Related Issue

N/A

Type of Change

  • ✨ New feature (non-breaking change that adds functionality)

Changes Made

  • tools/transcription_tools.py_transcribe_yandex() provider function;
    three helper functions (_get_audio_duration_seconds, _get_audio_channels,
    _ensure_mono_for_yandex); wired into _get_provider() explicit check and
    auto-detect fallback chain; added to transcribe_audio() dispatcher
  • tests/tools/test_transcription_tools.py — 22 new unit tests covering the
    provider function, helper functions, dispatcher integration, and auto-detect
    logic; also adds the missing XAI_API_KEY delenv to the clean_env fixture
  • cli-config.yaml.example — documents the stt.yandex config sub-section
    and updates the provider list comment (which was missing xAI too)

How to Test

  1. Set YANDEX_API_KEY=<your key> in ~/.hermes/.env
  2. Send a voice message on Telegram — the gateway will transcribe it via
    Yandex SpeechKit automatically (auto-detect picks Yandex when no local
    faster-whisper is installed)
  3. Or force it explicitly: add stt: { provider: yandex } to ~/.hermes/config.yaml
  4. Run unit tests: pytest tests/tools/test_transcription_tools.py -v -k "Yandex or yandex or EnsureMono or AudioChannel or AudioDuration"

Expected: all 22 new tests pass without a real API key (all HTTP calls are mocked).

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (feat(stt):)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes
  • I've tested on my platform: Ubuntu 24.04

Documentation & Housekeeping

  • I've updated cli-config.yaml.example with the new stt.yandex config keys
  • N/A — no CONTRIBUTING.md or AGENTS.md architecture changes
  • Cross-platform: all open() calls use binary mode; subprocess.run uses list form with check=False; no POSIX-only signals or file modes used
  • N/A — tool schemas unchanged (STT is internal, not exposed as an LLM tool)

Add Yandex SpeechKit Sync API as a seventh STT provider.

Features:
- Raw binary upload to POST stt.api.cloud.yandex.net/speech/v1/stt:recognize
- Native OGG/Opus support (Telegram and WhatsApp default voice format)
- WAV/PCM support with sample-rate validation (8000/16000/48000 Hz)
- Automatic stereo-to-mono downmix via ffmpeg when needed
- File size guard (1 MB hard limit) and duration guard (30s hard limit)
- Helper functions: _get_audio_duration_seconds(), _get_audio_channels(),
  _ensure_mono_for_yandex() — each degrades gracefully when ffprobe/ffmpeg
  is not installed
- Language, topic, folder_id, profanity_filter, raw_results config options
- Full integration: explicit config + auto-detect (lowest priority, after xAI)

Config (config.yaml):
  stt:
    provider: yandex
    yandex:
      language: ru-RU          # BCP-47 language code (default: ru-RU)
      topic: general           # general | general:rc | general:deprecated
      folder_id: ""            # Yandex Cloud folder ID (optional)
      profanity_filter: false  # Filter profanity (default: false)
      raw_results: false       # numbers as words (true) or digits (false)

Requires YANDEX_API_KEY environment variable.
Auto-detect priority: local > groq > openai > mistral > xai > yandex > none
@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/tools Tool registry, model_tools, toolsets tool/tts Text-to-speech and transcription labels May 11, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Thanks for the thorough implementation and focused test coverage.

This automated hermes-sweeper review is closing this under the standing in-tree-provider-integration policy: third-party vendor integrations should ship as standalone plugins rather than adding vendor-specific code to the core tree.

  • The PR adds the Yandex-specific handler directly to tools/transcription_tools.py:871 (ff3364c148d601f929f3095ec453ffe130b1ab8a).
  • Current main already provides the intended route: TranscriptionProvider explicitly supports new STT backends without modifying tools/transcription_tools.py (agent/transcription_provider.py:22-35), and transcribe_audio() dispatches non-built-in providers through that plugin surface (tools/transcription_tools.py:1713-1737).
  • Please publish this as a standalone Yandex SpeechKit STT plugin using ctx.register_transcription_provider(); users can install it under ~/.hermes/plugins/ or through a pip entry point.

Closed as not-planned per standing maintainer policy (in-tree-provider-integration). This is a design-direction decision, not a code-quality judgment — see the Contribution Rubric in AGENTS.md for what the project is looking for. If you believe this policy was misapplied to your change, comment here and a maintainer will take a look.

@teknium1 teknium1 closed this Jul 13, 2026
@teknium1 teknium1 added the sweeper:not-planned Sweeper: closed per standing maintainer policy (design direction) label Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/tools Tool registry, model_tools, toolsets P3 Low — cosmetic, nice to have sweeper:not-planned Sweeper: closed per standing maintainer policy (design direction) tool/tts Text-to-speech and transcription type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants