Skip to content

fix(gateway): transcribe configured Discord audio attachments - #31225

Open
mgreez wants to merge 1 commit into
NousResearch:mainfrom
mgreez:fix/discord-audio-attachment-stt-upstream
Open

fix(gateway): transcribe configured Discord audio attachments#31225
mgreez wants to merge 1 commit into
NousResearch:mainfrom
mgreez:fix/discord-audio-attachment-stt-upstream

Conversation

@mgreez

@mgreez mgreez commented May 24, 2026

Copy link
Copy Markdown

What does this PR do?

Related Issue

Fixes #

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

How to Test

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform:

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A

For New Skills

  • This skill is broadly useful to most users (if bundled) — see Contributing Guide
  • SKILL.md follows the standard format (frontmatter, trigger conditions, steps, pitfalls)
  • No external dependencies that aren't already available (prefer stdlib, curl, existing Hermes tools)
  • I've tested the skill end-to-end: hermes --toolsets skills -q "Use the X skill to do Y"

Screenshots / Logs

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/gateway Gateway runner, session dispatch, delivery platform/discord Discord bot adapter tool/tts Text-to-speech and transcription labels May 24, 2026

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for preserving the default file-attachment behavior and making the proposed STT route channel-scoped.

Problems

  • The changed MessageType.AUDIO branch still routes every path in event.media_urls to STT. Current main has per-attachment MIME classification in gateway/run.py:2024-2029; a mixed Discord message whose first attachment is audio can also contain image/document paths, and those must not be transcribed. See tests/gateway/test_mixed_attachment_routing.py:55-59.
  • tests/gateway/test_discord_audio_attachment_stt.py tests the predicate but not the live _prepare_inbound_message_texttranscribe_audio path, so the configured and unconfigured end-to-end behaviors are not covered.
  • The configuration bridge has moved to gateway/config.py:1156-1235, and the Discord adapter is now under plugins/platforms/discord/adapter.py; this needs a current-main salvage. Please also document the new user-facing setting near website/docs/user-guide/messaging/discord.md:310-335.

Suggested changes

  • Use _event_media_is_audio(event, i) for the opt-in routing and add mixed-media regression coverage.
  • Add configured/unconfigured inbound-pipeline tests with mocked transcribe_audio.

Automated hermes-sweeper review.

Comment thread gateway/run.py
# capture channel can opt in via
# discord.transcribe_audio_attachment_channels so iPhone/Discord
# voice notes that arrive as generic audio attachments still get
# transcribed.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please classify each attachment before appending it here. An AUDIO-typed Discord message can contain mixed media; routing every event.media_urls entry through STT would send accompanying images/documents to transcribe_audio. Current main provides _event_media_is_audio(event, i) for this per-MIME distinction.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 13, 2026
@andrexibiza

Copy link
Copy Markdown
Contributor

Vox Lockin lane 07 probe — feature absent on main, PR stale/conflicting; not part of the delivery-reliability class.

Probe against origin/main @ 70db671fac:

Recommendation: rebase and re-open for review as a feature; the diff is clean and the tests (test_discord_audio_attachment_stt.py) look solid. No duplicate PR opened.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists platform/discord Discord bot adapter sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades tool/tts Text-to-speech and transcription type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants