fix(gateway): show 🎙 transcript prefix in streaming mode - #8644
Closed
maiixu wants to merge 9 commits into
Closed
Conversation
Add Amazon Bedrock as a first-class inference provider using the Converse API via boto3. Supports three authentication methods: Bedrock API keys (Bearer tokens), AWS profiles (~/.aws/credentials), and AWS access key pairs. - agent/bedrock_adapter.py — Converse API adapter (message conversion, tool schema translation, response normalization) - hermes_cli/auth.py — ProviderConfig, _resolve_bedrock_base_url(), region-aware endpoint construction, provider aliases - hermes_cli/providers.py — HermesOverlay with bedrock_converse transport, aliases, labels - hermes_cli/models.py — Curated model list with global/eu/us inference profile IDs and bare model IDs - hermes_cli/config.py — AWS_BEARER_TOKEN_BEDROCK and AWS_BEDROCK_REGION env vars - hermes_cli/runtime_provider.py — bedrock_converse in valid API modes - agent/models_dev.py — PROVIDER_TO_MODELS_DEV mapping - New api_mode 'bedrock_converse' with full lifecycle support: init, switch_model, _build_api_kwargs, _interruptible_api_call, response validation, finish_reason extraction, response normalization - Safe vars() call in error logging (handles dict responses from boto3) - Dedicated _model_flow_bedrock() with interactive setup: - Auth picker: existing API key, new API key, AWS profile, access keys - Region selector with common regions + custom input - Model selection from curated list - Persists api_mode=bedrock_converse in config.yaml - Added boto3 to pyproject.toml Bedrock models use cross-region inference profile prefixes: - global. — works from any region - eu. — EU cross-region routing - us. — US cross-region routing - bare — single-region direct access
* stt: inject hotwords as initial_prompt for local/groq/openai Reads ~/.hermes/stt_hotwords.txt and passes terms as initial_prompt (faster-whisper) or prompt (Groq/OpenAI API). File read fresh each call. * gateway: prepend voice transcript 🎙 「...」 to response Store transcript in self._pending_voice_transcript during enrichment. Before returning the response to the user, prepend 🎙 「transcript」. LLM still receives the original [The user sent a voice message~] format. No LLM tokens spent on formatting — done at gateway layer.
Conflict resolution left XIAOMI_BASE_URL dict unclosed when merging Bedrock commit — caused SyntaxError on gateway startup.
auth.py: ProviderConfig(xiaomi) was missing closing ) providers.py: HermesOverlay(xiaomi) was missing closing )
- config.py: XIAOMI_BASE_URL dict missing closing } - auth.py: xiaomi ProviderConfig missing closing ) - providers.py: xiaomi HermesOverlay missing closing ) - run.py: f-string had literal newlines, replaced with \n escape
…sync
By default both skills_sync (upstream bundled) and skill_manager (agent-
created) write to the same ~/.hermes/skills/. This makes it impossible
to keep upstream bundled skills and agent/user-created skills in separate
directories without inspecting .bundled_manifest.
Add skills.create_dir config key (default: empty = ~/.hermes/skills/).
When set, new skills created via skill_manage() go there instead.
This lets users configure:
skills:
create_dir: ~/personal-intelligence/hermes/skills # agent-created here
external_dirs:
- ~/personal-intelligence/hermes/bundled-skills # upstream bundled here
Also update get_all_skills_dirs() to include create_dir in the read path
so created skills are discoverable alongside bundled and external ones.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
feat(skills): add skills.create_dir to separate creation from upstream sync
Previously the reset was inside `if event.media_urls:`, so a typed message following a voice message never cleared the field. The old transcript was prepended again (🎙 「…」) to every subsequent reply until another media message arrived. Move the reset to before the media_urls check so it is unconditional. Co-authored-by: claude-bot-maiixu[bot] <275422726+claude-bot-maiixu[bot]@users.noreply.github.com> Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
The voice transcript prefix (🎙 「…」) was only added in the non-streaming path (line 3632). When streaming was active the agent sent `already_sent=True`, triggering an early return that skipped the prefix entirely — so voice message replies never showed the transcript header in streaming mode. Fix: seed the transcript as the first delta on the stream consumer right after it is created. The prefix is included in the first edit just like any other accumulated text, keeping behaviour consistent between streaming and non-streaming paths. Adds two regression tests in TestVoiceTranscriptPrefix. Co-authored-by: Mai Xu <maizehsu02@gmail.com> Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The voice transcript prefix (
🎙 「…」) was silently dropped whenever streaming was active.Root cause
The prefix is added at run.py:3632:
But in streaming mode the agent sets
already_sent=True, which triggers an earlyreturn Noneat line 3623 — before line 3632 is ever reached. PR #4 fixed the "transcript leaking into subsequent messages" bug but missed this path entirely.Fix
After creating the
GatewayStreamConsumer, seed the transcript as the first delta:The prefix enters the stream consumer's queue first, so the initial edit already contains the
🎙 「…」header before any agent tokens arrive. The non-streaming path (line 3632) is unchanged — the two paths are mutually exclusive.Test
Added
TestVoiceTranscriptPrefixintest_stream_consumer.py:test_transcript_prefix_in_first_message— seeded transcript appears at the start of the first sent messagetest_no_prefix_for_text_messages— no🎙in non-voice messages