fix(matrix): restore voice bubbles — send_voice rejected is_voice, dr… - #99712
simonamsharp wants to merge 1 commit into
Conversation
…opping MSC3245
Matrix voice notes are currently broken in two independent ways.
1. `BasePlatformAdapter` media delivery calls
`self.send_voice(..., is_voice=is_voice)` (gateway/platforms/base.py), but
`MatrixAdapter.send_voice()` accepts no such parameter and has no
`**kwargs`. Every audio attachment therefore raises
TypeError: MatrixAdapter.send_voice() got an unexpected keyword
argument 'is_voice'
and is logged as "[Matrix] Error sending media". `_send_local_file()`
already accepts `is_voice`, so the parameter is added and forwarded.
2. Element only draws an inline player for an `m.audio` event carrying the
MSC3245 `org.matrix.msc3245.voice` flag, which is set only when
`is_voice` is true. `is_voice` comes from an `[[audio_as_voice]]`
directive in the response text, which `GatewayRunner` auto-appends from
tool results — but only `if "MEDIA:" not in final_response`. An agent that
synthesises its own audio and emits its own `MEDIA:` tag suppresses that
block entirely and loses the directive, so its voice notes arrive as file
cards.
Fixes:
- `MatrixAdapter.send_voice()` takes `is_voice` (default True) plus
`**kwargs` and forwards it, and forces it True for `.ogg`/`.oga`/`.opus` —
that is precisely the container MSC3245 voice messages use, so the flag
should not depend on the caller having inferred intent. Non-Ogg input still
honours the caller, so a plain `.mp3` attachment is not turned into a voice
note.
- `gateway/run.py` recovers the `[[audio_as_voice]]` directive on its own when
the agent wrote its own `MEDIA:` tag, without re-appending the MEDIA tags
(which would deliver the file twice).
Deciding this in code rather than relying on the model to emit a marker also
makes delivery deterministic, which matters on smaller local models that
follow such instructions unreliably.
Tests: 11 new cases in tests/gateway/test_matrix_voice_bubble_msc3245.py,
covering the TypeError, the flag in both directions (present when is_voice,
absent otherwise, so ordinary audio is not converted to voice notes), and the
directive-to-is_voice path. All 7 send_voice cases fail on main and pass with
this change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0196UNhqc3eiwuW7y4xaRHnx
alt-glitch
added
type/bug
The two layers of this fix are coherent. In the gateway, when the agent wrote its own Non-blocking: forcing Verdict: LGTM |
|
Consolidation map for send_voice is_voice kwargs vs BasePlatformAdapter (exact heads at comment time):
Same contract miss class. Prefer landing one Matrix fix + shared adapter **kwargs/is_voice acceptance pattern, then close overlaps as duplicate, rather than merging the spray independently. Happy to draft the shared helper if maintainers want consolidation over land-as-is. No competing PR from me. |
Voice / STT KEEP cluster map (comment-only)Preferred vehicle for the Matrix Cluster (gateway / TTS / platform voice)
Recommended order
No new mega-issue / Wave from this comment. No competing PR from me. |
4 participants
…opping MSC3245
Matrix voice notes are currently broken in two independent ways.
BasePlatformAdaptermedia delivery callsself.send_voice(..., is_voice=is_voice)(gateway/platforms/base.py), butMatrixAdapter.send_voice()accepts no such parameter and has no**kwargs. Every audio attachment therefore raisesand is logged as "[Matrix] Error sending media".
_send_local_file()already acceptsis_voice, so the parameter is added and forwarded.Element only draws an inline player for an
m.audioevent carrying the MSC3245org.matrix.msc3245.voiceflag, which is set only whenis_voiceis true.is_voicecomes from an[[audio_as_voice]]directive in the response text, whichGatewayRunnerauto-appends from tool results — but onlyif "MEDIA:" not in final_response. An agent that synthesises its own audio and emits its ownMEDIA:tag suppresses that block entirely and loses the directive, so its voice notes arrive as file cards.Fixes:
MatrixAdapter.send_voice()takesis_voice(default True) plus**kwargsand forwards it, and forces it True for.ogg/.oga/.opus— that is precisely the container MSC3245 voice messages use, so the flag should not depend on the caller having inferred intent. Non-Ogg input still honours the caller, so a plain.mp3attachment is not turned into a voice note.gateway/run.pyrecovers the[[audio_as_voice]]directive on its own when the agent wrote its ownMEDIA:tag, without re-appending the MEDIA tags (which would deliver the file twice).Deciding this in code rather than relying on the model to emit a marker also makes delivery deterministic, which matters on smaller local models that follow such instructions unreliably.
Tests: 11 new cases in tests/gateway/test_matrix_voice_bubble_msc3245.py, covering the TypeError, the flag in both directions (present when is_voice, absent otherwise, so ordinary audio is not converted to voice notes), and the directive-to-is_voice path. All 7 send_voice cases fail on main and pass with this change.
Claude-Session: https://claude.ai/code/session_0196UNhqc3eiwuW7y4xaRHnx
What does this PR do?
Related Issue
Fixes #
Type of Change
Changes Made
How to Test
Checklist
Code
fix(scope):,feat(scope):, etc.)pytest tests/ -qand all tests passDocumentation & Housekeeping
docs/, docstrings) — or N/Acli-config.yaml.exampleif I added/changed config keys — or N/ACONTRIBUTING.mdorAGENTS.mdif I changed architecture or workflows — or N/AFor New Skills
hermes --toolsets skills -q "Use the X skill to do Y"Screenshots / Logs