feat(line): add opt-in smart reply modality - #40933
Conversation
1e5f706 to
5683949
Compare
5683949 to
4e5a81a
Compare
|
Rebased this feature PR onto the refreshed adapter policy branch (#40931). Local verification: uv run --with pytest --with pytest-asyncio --with aiohttp python -m pytest \
tests/gateway/test_line_smart_modality.py \
tests/gateway/test_line_plugin.py \
tests/gateway/test_reply_delivery_policy.py \
tests/gateway/test_voice_command.py::TestAutoVoiceReply \
tests/gateway/test_media_download_retry.py \
-q -o 'addopts='
# 138 passed
uv run --with pytest --with pytest-asyncio --with aiohttp python -m pytest \
tests/gateway/test_voice_command.py \
-q -o 'addopts='
# 163 passed, 21 skipped |
a659756 to
40aa476
Compare
3797890 to
3bc90ec
Compare
3bc90ec to
da82cca
Compare
teknium1
left a comment
There was a problem hiding this comment.
Thanks for the LINE modality work. The feature is still absent from current main, but two delivery/documentation issues need resolution before salvage.
Problems
gateway/run.py:11401suppresses only the normal final send. Current main marksalready_sentafter the final streamed content is confirmed delivered (gateway/run.py:20065-20073), and aNoneresponse means the adapter does not send another message (gateway/platforms/base.py:4894-4914). Thus, with streaming enabled, the voice reply can be followed by text that was already streamed.plugins/platforms/line/plugin.yaml:67addsLINE_SMART_MODALITY, but neitherwebsite/docs/user-guide/messaging/line.md:159-177norwebsite/docs/reference/environment-variables.md:549-569documents it.
Suggested changes
- Gate/disable text streaming for smart voice-delivery turns before content is emitted, and add an integration test for the streaming-enabled path.
- Add the setting and its
LINE_PUBLIC_URLaudio-delivery prerequisite to both LINE documentation tables.
Automated hermes-sweeper review.
da82cca to
ce65e86
Compare
|
Thanks for the review — both points fixed; specifics in the inline replies, including two known limits of the streaming gate I'd rather disclose up front (interim-preview path, rapid-fire modality race). Rebased onto current main atop #40931. CI green, mergeable. |
9b117c7 to
384f6a5
Compare
384f6a5 to
5ab0632
Compare
Rebase note: upstream 73e193c ("fix(line): normalize inbound media types and cache routing") independently shipped a superset of this branch's adapter fix — typed cache helper dispatch, fileName threading, and a (path, media_type) return from _download_media. The adapter changes are therefore dropped in favor of upstream's version. What remains from the original commit: the mocked regression tests covering _download_media routing for all four LINE content types, cache/fetch failure fallbacks, and fileName threading from the message event — adapted to upstream's keyword-only ``filename=`` parameter and tuple return, and to media_types now carrying MIME types. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Rebase note (onto f75b577): upstream's voice.auto_tts sync (_should_auto_tts_for_chat, "voice accompanies text replies unless the chat explicitly set off", with unset voice_mode None distinct from "off") and the NousResearch#60671 streaming-TTS skip landed in these same paths since this branch's base. Adapted the graft to preserve both: the base adapter's default reply_delivery_policy and the runner's legacy fallback now include the auto_tts term, _reply_delivery_policy passes voice_mode through raw (None when unset) instead of defaulting to "off", and the runner call site keeps upstream's streaming-TTS guard while threading _voice_reply_sent/suppress-text through it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> [rebase note 2026-08-01: re-resolved against upstream e444d16 — adopted upstream's tightened voice.auto_tts precedence (fallback only when chat has no explicit voice mode, `voice_mode is None`, upstream changed from `!= "off"`) in both the base adapter's default reply_delivery_policy and the runner's legacy fallback]
self.adapters is the default profile's adapter map. Resolve the reply-delivery-policy and inbound-observe adapter through _adapter_for_source(event.source) so multiplex secondary profiles consult their own adapter's policy (aligns with 8a9bc38). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Catch exceptions from the adapter reply_delivery_policy callback and fall back to the legacy voice-mode gate so a buggy adapter policy can never disrupt final reply delivery. Mirrors the isolation pattern used by _observe_inbound_message. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
With streaming enabled, the final text is streamed and marked already_sent before the reply-delivery policy runs, so returning None after the voice reply could not retract it: users got a voice reply plus the duplicate streamed text. Add a BasePlatformAdapter.voice_reply_replaces_text pre-flight hook, implement it for LINE smart modality from the observed inbound modality, and consult it in both gateway streaming setups BEFORE any content is emitted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> [rebase note 2026-07-30] Upstream extracted the inline run_sync closure into TurnRunner.run_sync; the streaming gate that lived in the old closure is ported to TurnRunner.run_sync (via self._runner._suppress_text_streaming_for_voice, source from ctx.source). No behavior change intended.
Add the opt-in smart reply modality toggle to the LINE user-guide env-var table and the environment-variables reference, noting that successful audio delivery requires LINE_PUBLIC_URL. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
5ab0632 to
69cf023
Compare
|
Consolidation update — this PR is now the single vehicle for the former #35785 → #40931 → #40933 stack.
One thing worth a maintainer's deliberate pick: #87397 proposes a per-platform |
Summary
smart_modality/LINE_SMART_MODALITY.reply_delivery_policy) that the modality routing rides on, with exception isolation and multiplex-safe adapter resolution._download_mediadispatch), whichmaincurrently lacks.Self-contained — one approval lands the whole former stack
This PR was previously presented as the top of a 3-PR stack (#35785 → #40931 → this). Both former base PRs are now closed in favor of this one: their entire diffs were already carried here as patch-equivalent commits, so the three open PRs presented ~triple review surface for one change. History on the review feedback:
73e193c03(fix(gateway): inbound voice classification/routing for Feishu, DingTalk, LINE, QQ, Photon, WhatsApp, Weixin #73515) — what survives here is exactly the regression coveragemainstill lacks._adapter_for_source(event.source)(multiplex-safe), and the policy callback wrapped with a fall-back to the legacy voice gate so a policy failure can never disrupt final-reply delivery.voice_reply_replaces_textpre-flight hook (with an integration test), andLINE_SMART_MODALITYis documented in both env-var tables.Rebased onto current
main(2026-08-21); the staleAUTHOR_MAPhunk was dropped (that dict is frozen — the contributor mapping now lives incontributors/emails/).Related open work
#87397 proposes a per-platform
suppress_text_when_voiceconfig flag solving an overlapping problem ingateway/platforms/base.py. The policy seam here is the more general mechanism (per-adapter, exception-isolated, modality-aware); flagging so a maintainer can pick one mechanism deliberately rather than both landing.Test Plan
python -m py_compile gateway/run.py gateway/platforms/base.py plugins/platforms/line/adapter.pypython -m pytest tests/gateway/test_reply_delivery_policy.py tests/gateway/test_line_smart_modality.py tests/gateway/test_line_plugin.py tests/gateway/test_platform_base.py -q(141 passed on the rebased head)