Skip to content

fix(gateway): keep the STT echo ledger across pending-media merges - #67281

Closed
Frowtek wants to merge 1 commit into
NousResearch:mainfrom
Frowtek:fix/stt-echo-ledger-merge
Closed

fix(gateway): keep the STT echo ledger across pending-media merges#67281
Frowtek wants to merge 1 commit into
NousResearch:mainfrom
Frowtek:fix/stt-echo-ledger-merge

Conversation

@Frowtek

@Frowtek Frowtek commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

Summary

_invalidate_pending_stt_cache() (added in #67248) clears the gateway-side
transcription cache when merge_pending_message_event() folds a follow-up
message into a still-pending event, so the next transcription call picks up the
merged text and attachments. That part is correct.

It also clears _gateway_pending_stt_echo_sent. That flag is not derived state —
it records that the transcript was already delivered to the user. Dropping it
makes the re-run transcription echo the earlier notes a second time.

for attr in (
    "_gateway_pending_stt_text",
    "_gateway_pending_stt_transcripts",
    "_gateway_pending_stt_echo_sent",   # <-- delivery record, not a cache
):

Both merge branches are affected, including the text-only follow-up case where
no new audio arrived at all: the cache is invalidated, the same voice note is
transcribed again (a second paid STT call), and the same line is echoed again.

Reproduction

  1. Voice note arrives; the interrupt monitor transcribes it and echoes 🎙️ "hello"
  2. The user sends a follow-up while the turn is still pending, so it merges
  3. The drain path re-transcribes and echoes 🎙️ "hello" a second time

Against origin/main, with a second voice note merging in, the user receives:

🎙️ "hello"
🎙️ "hello"     <-- repeat of an already-delivered transcript
🎙️ "world"

Fix

Keep the ledger out of the invalidation set, and track it as a count of
already-echoed transcripts rather than a single boolean.

A count is what the merge case actually needs: re-running transcription over the
extended media list returns the earlier transcripts as a prefix of the new
list, so echoing only the unsent tail suppresses the repeat while still
surfacing a newly merged voice note. Two identical transcripts from two separate
notes stay distinct — a value-based dedup would have collapsed them.

scenario before after
text follow-up merges (no new audio) hello, hello hello
second voice note merges hello, hello, world hello, world
same phrase spoken in both notes hello, hello, hello hello, hello

_gateway_pending_stt_echo_sent had exactly one reader (the echo helper itself),
so replacing it with _gateway_pending_stt_echoed is self-contained.

Testing

Two regression tests added to tests/gateway/test_telegram_voice_v0_regressions.py,
driving the real merge_pending_message_event() and echo helper:

  • test_pending_stt_merge_does_not_re_echo_delivered_transcript
  • test_pending_stt_merge_echoes_only_the_newly_merged_transcript

Both fail on origin/main with the duplicate-echo symptom above and pass with
this change.

tests/gateway/test_telegram_voice_v0_regressions.py .... 8 passed

Full gateway voice/STT/pending/merge/telegram selection compared against a clean
origin/main worktree: identical 15 pre-existing failures (network/SSL-dependent
Telegram tests), 2002 -> 2004 passed, no new failures.

Checklist

  • Bug is reproducible on main and covered by a failing-before/passing-after test
  • No regressions in the surrounding suite (baseline-compared against origin/main)
  • Change is scoped to the defect — no unrelated refactoring
  • Tested on Ubuntu 24.04

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for isolating the pending-event STT cache/echo interaction. The reported defect is present on current main: gateway/platforms/base.py:2120 deletes _gateway_pending_stt_echo_sent, while gateway/run.py:16431-16434 uses that attribute as the one-time echo guard. The count-and-tail approach addresses the merged-media prefix correctly.

Problems

  • The PR also carries an unrelated api_content replay hunk in gateway/run.py:852-868. Current main already has that exact code from 7b3dcee928e6; it should not be bundled with this STT fix.
  • The tests do not cover the documented identical-transcript case. A value-based deduplication implementation could pass the added hello/world test while incorrectly collapsing two separate hello voice notes.

Suggested changes

  • Drop the already-landed replay-sidecar hunk.
  • Add a two-voice-note, same-transcript regression asserting two echoes.

Automated hermes-sweeper review.

Comment thread gateway/run.py
@@ -849,6 +849,23 @@ def _build_replay_entry(
providers.
"""
entry: Dict[str, Any] = {"role": role, "content": content}
# api_content sidecar (persist-what-you-send, prompt-cache stability):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This api_content replay hunk is unrelated to the STT ledger and is already present on current main at gateway/run.py:852-868 via 7b3dcee928e6. Please remove it from this PR so the salvage remains scoped to the STT fix.

@alt-glitch alt-glitch added type/bug Something isn't working comp/gateway Gateway runner, session dispatch, delivery tool/tts Text-to-speech and transcription platform/telegram Telegram bot adapter P2 Medium — degraded but workaround exists needs-decision Awaiting maintainer decision before any implementation sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) labels Jul 19, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Related to merged #67248 (pending-STT cache invalidation) and #67274 (already-merged api_content cache work). The STT ledger repair is distinct, but this branch should rebase before review so the redundant cache hunk is dropped.

@teknium1 teknium1 added the sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit label Jul 19, 2026
_invalidate_pending_stt_cache() clears the gateway-side transcription
cache when merge_pending_message_event() folds a follow-up message into a
still-pending event, so the next transcription picks up the merged text
and attachments.  It also cleared _gateway_pending_stt_echo_sent, but that
flag is not derived state — it records that the transcript was already
delivered to the user.

Dropping it makes the re-run transcription echo the earlier notes a second
time.  Both merge branches are affected, including the text-only follow-up
case where no new audio arrived at all: there the cache is invalidated,
the same voice note is transcribed again (a second paid STT call) and the
same line is echoed again.

Sequence:

  1. voice note arrives, interrupt monitor transcribes it and echoes
     '🎙️ "hello"'
  2. user sends a follow-up while the turn is still pending, so it merges
  3. drain path re-transcribes and echoes '🎙️ "hello"' a second time

Keep the ledger out of the invalidation set and track it as a count of
already-echoed transcripts instead of a single boolean.  A count is what
the merge case actually needs: re-running transcription over the extended
media list returns the earlier transcripts as a prefix of the new one, so
echoing only the unsent tail suppresses the repeat while still surfacing a
newly merged voice note.  A count rather than a set of seen values, so two
separate notes that transcribe identically stay two distinct deliveries —
covered by test_pending_stt_merge_echoes_two_identical_transcripts.

The guard stays within the 12-line window that
test_all_gateway_transcript_echo_sends_are_gated enforces over run.py.
@Frowtek
Frowtek force-pushed the fix/stt-echo-ledger-merge branch from 667aaff to 91cba28 Compare July 19, 2026 13:19
teknium1 added a commit that referenced this pull request Jul 28, 2026
…point

Follow-up for salvaged #65023/#53020: _prepare_busy_steer_text now calls
_transcribe_and_echo_pending_voice (the same helper the interrupt monitor
and pending-drain paths use) instead of a private transcription+echo copy,
so out-of-band voice pays one STT call per platform message and the echo
respects the count-based ledger from #67281. can_steer now accepts events
whose attachments are all STT-eligible voice media, completing the steer
half of #58780. Adds extract_media gating tests for #44826 and the
contributor mapping for chefboyrdave21.
@teknium1

Copy link
Copy Markdown
Contributor

Merged into main via consolidated salvage PR #73518 (merge ea80b557ae). Your STT echo-ledger fix (keep it out of _invalidate_pending_stt_cache, count-based tail echo) was cherry-picked as 92818ae with your authorship.

Your contribution is credited to you in git history. Thank you! Closing this PR as merged-via-salvage.

@teknium1 teknium1 closed this Jul 29, 2026
randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
…point

Follow-up for salvaged NousResearch#65023/NousResearch#53020: _prepare_busy_steer_text now calls
_transcribe_and_echo_pending_voice (the same helper the interrupt monitor
and pending-drain paths use) instead of a private transcription+echo copy,
so out-of-band voice pays one STT call per platform message and the echo
respects the count-based ledger from NousResearch#67281. can_steer now accepts events
whose attachments are all STT-eligible voice media, completing the steer
half of NousResearch#58780. Adds extract_media gating tests for NousResearch#44826 and the
contributor mapping for chefboyrdave21.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery needs-decision Awaiting maintainer decision before any implementation P2 Medium — degraded but workaround exists platform/telegram Telegram bot adapter sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages tool/tts Text-to-speech and transcription type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants