Skip to content

fix(stt): reject structured transcription errors (#78098) - #78118

Closed
RerankerGuo wants to merge 1 commit into
NousResearch:mainfrom
RerankerGuo:fix/issue-78098-stt-error-repr
Closed

RerankerGuo wants to merge 1 commit into
NousResearch:mainfrom
RerankerGuo:fix/issue-78098-stt-error-repr

Conversation

@RerankerGuo

@RerankerGuo RerankerGuo commented Aug 4, 2026

Copy link
Copy Markdown

What does this PR do?

Fixes #78098 by preventing structured STT error objects from becoming user
message text.

An OpenAI-compatible transcription response can be a structured object with
text=None and an explicit provider error. Current main falls through to
str(transcription), treats the Python representation as a successful
transcript, and sends it as the user's message.

Structured responses now:

  • raise their explicit provider error through the existing transcription
    failure path; or
  • return an empty transcript when they contain neither text nor an error.

Plain strings and unknown scalar response types preserve existing behavior.

Current-main reproduction

Verified against origin/main at 87fd0ed25 using the same two regression
tests added by this PR:

scripts/run_tests.sh tests/tools/test_transcription_tools.py \
  -k 'openai_transcription_error_object_is_not_returned_as_text or rejects_structured_error_instead_of_stringifying_repr' -q

# current main production code: 2 failed
# this PR: 2 passed

On current main, _transcribe_openai() reports success=True, and
_extract_transcript_text() does not raise for the structured provider error.

End-to-end validation

Independent native validation on this PR traced the reported object through
_extract_transcript_text -> _transcribe_openai -> the web transcription
endpoint -> Desktop conversation mode. The PR converts that path to the
existing HTTP 400/error-toast flow while preserving successful plain-string,
object-with-text, dict-with-text, and silent-empty responses.

Verification

scripts/run_tests.sh tests/tools/test_transcription_tools.py \
  -k 'openai_transcription_error_object_is_not_returned_as_text or rejects_structured_error_instead_of_stringifying_repr or ExtractTranscriptText' -q
# 4 passed

uv run ruff check tools/transcription_tools.py tests/tools/test_transcription_tools.py
# All checks passed

.venv/bin/python scripts/check-windows-footguns.py --diff origin/main
# No Windows footguns found

git diff --check
# passed

The complete transcription test file reports 51 passed and one unrelated
faster-whisper configuration assertion failure. That exact failure reproduces
on current main (compute_type is int8, while the stale assertion expects
float32).

Scope

  • No Desktop-side change is required.
  • No provider routing or model selection behavior changes.
  • No change to valid transcript normalization.

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists tool/tts Text-to-speech and transcription labels Aug 4, 2026
@andrexibiza

Copy link
Copy Markdown
Contributor

Adversarial verification (Vox Lockin lane 09 — desktop voice class; reviewed for #78098):

Verdict: mergeable and fixes the user-visible symptom — no duplicate PR opened.

Verified against origin/main @ 70db671 and PR head fbf8d0f276:

  1. Leak path confirmed on main: _extract_transcript_text (tools/transcription_tools.py:2676-2677) falls back to str(transcription) when a structured response has text=None. An OpenAI-compatible STT server returning an error-shaped Transcription(text=None, logprobs=None, usage=None, error='Transcription failed') object in a 200 envelope produces exactly the repr the issue's agent.log shows (the "81 chars" logged on main is the repr length). It then flows success: True/api/audio/transcribe → desktop transcribeVoiceAudio → submitted as the message.
  2. Error path: with this PR, the structured error raises ValueError_transcribe_openai returns success: False with the real error → web server HTTP 400 → desktop shows the transcription-failed toast and re-listens. The repr can never become a transcript or message.
  3. Success path preserved (probed directly on PR head): plain-string, object-with-text, and dict-with-text responses all still extract normally; structured empty responses return "" (silence → re-listen), never the repr.
  4. Tests: tests/tools/test_transcription_tools.py on PR head — 51 passed / 1 failed, and the single failure (TestRunCommandSttIdleTimeout::test_stderr_progress_extends_beyond_timeout) fails identically on pristine main (pre-existing Windows timing flake, unrelated).
  5. Merge-clean against current main (git merge-tree clean).

No desktop-side change is needed: the API contract is now enforced server-side and the desktop's onTranscribeAudio error path already handles rejection correctly. Please go ahead and merge.

Do not stringify SDK transcription objects with missing text into user messages. Surface explicit provider errors through the existing failure path and treat empty structured responses as empty transcripts.
@RerankerGuo
RerankerGuo force-pushed the fix/issue-78098-stt-error-repr branch from 85cbbe2 to b8e379b Compare August 8, 2026 12:13
@RerankerGuo

Copy link
Copy Markdown
Author

@kshitijk4poor, would you be willing to take a focused look when bandwidth permits? Tagging you because you recently maintained the STT path in #77516 and #81027.

This is a 2-file fix for P2 issue #78098: 13 production lines prevent structured provider errors from becoming user messages. The same two regressions fail on current main and pass at b8e379b668.

An independent verifier also traced the full failure path through the desktop endpoint and confirmed valid string/object/dict transcripts remain unchanged. CI has not failed; the fork workflow is awaiting maintainer approval.

If another owner is more appropriate, no action is needed. I will not re-ping.

@RerankerGuo RerankerGuo closed this by deleting the head repository Aug 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Medium — degraded but workaround exists tool/tts Text-to-speech and transcription type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Desktop conversation mode drops successful transcription and sends Transcription(...) error repr as the message

3 participants