fix(stt): provider robustness — device config, CUDA fallbacks, container normalization (SILK/m4a), lazy-install recovery - #73510
Merged
Conversation
The local STT transcription function hardcoded device="auto" and compute_type="auto" when instantiating WhisperModel, ignoring the user's stt.local.device and stt.local.compute_type config values. Closes #8319
- Add Blackwell-specific cuBLAS error marker to _CUDA_LIB_ERROR_MARKERS - Allows CPU fallback on RTX 5090 (sm_120) when faster-whisper reports CUBLAS_STATUS_NOT_SUPPORTED instead of loading successfully - Add regression test for CUBLAS_STATUS_NOT_SUPPORTED path Closes #17526
Replace aise ImportError(str(e)) with pass in the except Exception handler of _import_edge_tts(), _import_elevenlabs(), and _import_mistral_client() so packages installed via PYTHONPATH or Docker layered filesystems still work when lazy_deps.ensure() raises. Also fix the Mistral STT path in transcription_tools.py which only caught ImportError, not FeatureUnavailable. Adds 6 regression tests using sys.modules fixtures (no builtins.__import__ patching).
Add isolated test where ensure('stt.mistral') raises FeatureUnavailable
but the raw mistralai.client.Mistral import succeeds, verifying the
transcription_tools.py fallthrough path introduced in the same PR.
Normalize the structured <asr_text> marker after extracting text from string, SDK object, and dictionary transcription responses. Preserve the current provider-aware STT configuration architecture. Refreshes #8773 on current main. Co-authored-by: angelos <angelos@oikos.lan.home.malaiwah.com> Assisted-by: Codex:gpt-5.6
The /voice status command only checked for 'local', 'groq', and 'openai' providers. Any other valid provider (local_command, mistral, xai, elevenlabs, or custom command providers) fell through to the generic MISSING message — even when transcription worked perfectly. - Import _has_any_command_stt_provider (already defined, never imported) - Add elif branches for local_command, mistral, xai, elevenlabs - Add generic catch-all via _has_any_command_stt_provider() for arbitrary custom command providers
PR review feedback: - Replace _has_any_command_stt_provider() with selected-provider check via _resolve_command_stt_provider_config() - Add _check_plugin_stt_provider() for plugin-registered backends - Add tests: selected command, unrelated command (should NOT pass), and plugin provider path
Log lazy-install failures at WARNING instead of DEBUG, with actionable guidance about venv write-permission issues (the most common cause of silent STT failures). Salvaged from PR #46127 (transcription_tools half only — the gateway DM hunks are superseded by main's neutral-marker enrichment design, and the Docker/CI files were unrelated scope). (cherry picked from commit d3e07bd, reduced)
Decode WeChat/QQ SILK v3 voice notes to WAV inside transcribe_audio so any platform that caches a .silk file gets STT for free (same central- normalization philosophy as the outbound container repair). pilk is lazy-installed on first use (stt.silk in tools/lazy_deps.py) instead of being added to the voice extra. Fixes the inbound half of #32196. (cherry picked from commit e5db793; reworked to compose with the provider-scoped upload size cap and to lazy-dep pilk)
… audio container Newer OpenAI transcription models (gpt-4o-transcribe, gpt-4o-mini-transcribe) reject some containers the legacy whisper-1 endpoint accepted -- notably the Ogg/Opus voice notes messaging platforms deliver -- returning a 400 'corrupted or unsupported' error, so voice-note transcription fails for users on those models even though SUPPORTED_FORMATS still advertises .ogg/.aac/.flac. Wrap the OpenAI upload: on a format-related BadRequestError, transcode the source to a compact 16 kHz mono AAC .m4a via ffmpeg and retry once. This is model-agnostic (no per-model format table to maintain) and adds no cost for formats the endpoint already accepts. Fixes #68719
…e STT Two small fresh fixes on top of the salvage wave: - Wrap the check-then-load of the module-global faster-whisper model in a double-checked threading.Lock so concurrent voice messages can't both download/load the model (#24767). - Treat an empty stt.openai.api_key as no-auth when stt.openai.base_url points at a loopback/RFC-1918/.local host, so local OpenAI-compatible STT servers (faster-whisper-server, speaches, vLLM whisper) work without a sham api_key value. Reimplements the idea from PR #25193 — credit @nnnet. Co-authored-by: nnnet <nnnet@users.noreply.github.com>
…registration errors Follow-ups for the salvaged wave: the auto-detect legacy-error test now stubs the split validators, the unknown-command-provider test expects the new provider_not_registered error, and _transcribe_local tolerates a null stt.local config section again.
_transcribe_openai now imports BadRequestError for the container-retry path; the managed-gateway fake module needs to provide it.
…ows footgun lint)
Contributor
૮ >ﻌ< ა ci reviewran on fa5bf44 ℹ️ InfoDesktop E2E visual evidence · View test artifacts · View job1 visual diff. inline evidence upload failed. Failed to upload diff-665a0833239e-onboarding-overlay-diff.png with gh image (exit code 1): Error uploading /home/runner/work/_temp/e2e-evidence/diff-665a0833239e-onboarding-overlay-diff.png: step 0 (get upload token): uploadToken not found on repo page — do you have write access to NousResearch/hermes-agent? (or, if NousResearch enforces SAML SSO, authorize at https://github.com/orgs/NousResearch/sso) |
alt-glitch
added
type/bug
teknium1
added a commit
that referenced
this pull request
Jul 28, 2026
…ntry point The rebase onto #73510's prepare/dispatch split left the guard inside _transcribe_prepared_audio, where source validation ran first and a blocked .env surfaced a format error instead of the read-block message. Guard now fires before any validation/preprocessing.
teknium1
added a commit
that referenced
this pull request
Jul 28, 2026
…ntry point The rebase onto #73510's prepare/dispatch split left the guard inside _transcribe_prepared_audio, where source validation ran first and a blocked .env surfaced a format error instead of the read-block message. Guard now fires before any validation/preprocessing.
teknium1
added a commit
that referenced
this pull request
Jul 29, 2026
…ntry point The rebase onto #73510's prepare/dispatch split left the guard inside _transcribe_prepared_audio, where source validation ran first and a blocked .env surfaced a format error instead of the read-block message. Guard now fires before any validation/preprocessing.
teknium1
added a commit
that referenced
this pull request
Jul 29, 2026
…ntry point The rebase onto #73510's prepare/dispatch split left the guard inside _transcribe_prepared_audio, where source validation ran first and a blocked .env surfaced a format error instead of the read-block message. Guard now fires before any validation/preprocessing.
This was referenced Jul 29, 2026
Closed
randlee
pushed a commit
to randlee/hermes-agent
that referenced
this pull request
Aug 11, 2026
…ntry point The rebase onto NousResearch#73510's prepare/dispatch split left the guard inside _transcribe_prepared_audio, where source validation ran first and a blocked .env surfaced a format error instead of the read-block message. Guard now fires before any validation/preprocessing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
14 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fix(stt): consolidated STT provider correctness & robustness wave (13 fixes)
Consolidated salvage branch for the Cluster-F STT provider backlog. All fixes live in
tools/transcription_tools.py(+tools/voice_mode.py,tools/lazy_deps.py, onegateway/run.pyimport guard) with per-contributor commits and original authorshippreserved via cherry-pick. Every premise re-verified against current main
(
f228e145ba, post-#73067) before picking. 389 tests pass across the touched testfiles; the CUBLAS regression test was sabotage-verified (fails when the marker is
removed).
What's included (per-commit)
stt.local.device/stt.local.compute_typefrom configauto/auto; CUDA→CPU fallback retained. Closes #8319CUBLAS_STATUS_NOT_SUPPORTEDadded to_CUDA_LIB_ERROR_MARKERSKMP_DUPLICATE_LIB_OKon Darwin arm64/translated. Numpy-pin hunk dropped (stale). Closes #28622lazy_deps.ensurefails (PYTHONPATH installs)gateway/run.pyno longer crashes message handling on ImportErrorerror_type: provider_not_registeredinstead of generic "No STT provider". Closes #55167language …<asr_text>envelopecheck_voice_requirements()recognizes all STT providers.silk(WeChat/QQ) voice-note preprocessingtranscribe_audio— any platform caching.silkgets STT free. Reworked:pilk==0.2.4is now a lazy dep (stt.silk) instead of avoice-extra pin. Fixes inbound half of #32196BadRequestError, ffmpeg → 16 kHz mono m4a, retry once. Composes with the language-hint kwargs on main. Fixes #68719base_urlthreading.Lock(#24767); emptystt.openai.api_keyaccepted for loopback/RFC-1918/.localbase URLs (no more shamapi_key: not-needed). Post-merge: close #25193 (dirty branch, reimplemented)Design notes
container retry both live at the shared chokepoints (
transcribe_audiopre-dispatchand
_transcribe_openai, which also backs DeepInfra and other OpenAI-compatibleendpoints) — mirroring the outbound
_repair_ogg_containerphilosophy from fix(tts): class-level .ogg container repair + multi-platform Opus voice detection #73072.validation were merged so
.silksources are capped before the decoder runs, whileother local-provider inputs skip the remote-upload cap entirely.
_resolve_stt_language()resolver untouched; thelanguage hint now also applies to the m4a retry path.
Testing
pytest -o addopts="" -qover 11 touched test files: 389 passed.CUBLAS_STATUS_NOT_SUPPORTEDfrom the marker tuple failstest_cublas_status_not_supported_retries_on_cpu; restoring it passes.local base_url resolution (loopback/private/public matrix), plus all tests carried
by the salvaged commits.
contributors/emails/for allnon-auto-resolving authors.
Post-merge actions
fix(stt): check_voice_requirements() should recognize all STT providers #41657 fix(stt): better error logging and smarter DM when STT lazy-install fails #46127 transcription: transcode to m4a and retry when OpenAI STT rejects the audio container #68732 fix(stt): preprocess .silk voice notes before transcription #11593 fix(stt): treat empty api_key as no-auth when base_url is local #25193
Infographic