Skip to content

feat(gateway): interactive controls with Telegram inline keyboards (Phase 1 of #503) - #2188

Closed
gaelcado wants to merge 4 commits into
NousResearch:mainfrom
gaelcado:pr/telegram-inline-support
Closed

feat(gateway): interactive controls with Telegram inline keyboards (Phase 1 of #503)#2188
gaelcado wants to merge 4 commits into
NousResearch:mainfrom
gaelcado:pr/telegram-inline-support

Conversation

@gaelcado

@gaelcado gaelcado commented Mar 20, 2026

Copy link
Copy Markdown

Phase 1 of #503 — Transport-Agnostic Interactive Controls

Implements the shared component model discussed in #503 — a transport-agnostic ControlsDef/ButtonDef abstraction that any platform adapter can render natively, with Telegram inline keyboards as the first native renderer.

What this adds

  • ControlsDef / ButtonDef TypedDicts — transport-agnostic controls model in gateway/platforms/base.py
  • render_controls_as_text() fallback — platforms without native support automatically get text-based buttons
  • send_interactive() API — base method with text fallback, Telegram override for InlineKeyboardMarkup
  • Clarify + Approval lifecycle — controls parameter on clarify tool (120s timeout), inline accept/reject for approvals
  • _pending_clarifications session tracking — maps callback queries to pending clarify futures
  • Telegram CallbackQueryHandler — handles button presses, routes to correct session
  • Cleanup on /stop and shutdown — pending interactions are cancelled gracefully

Key design decisions

  • Additive only — zero breaking changes. All existing platform adapters get automatic text fallback.
  • Transport-agnostic model — Discord, Slack, and WhatsApp can adopt native rendering later by overriding send_interactive()
  • MAX_INTERACTIVE_BUTTONS — per-adapter limit (Telegram: 8, WhatsApp: 3) to respect platform constraints

Commit structure (reviewable incrementally)

  1. feat(gateway): add transport-agnostic ControlsDef/ButtonDef controls model — foundation types and fallback rendering
  2. feat(gateway): wire interactive clarify/approval lifecycle with Telegram inline keyboards — lifecycle integration and Telegram adapter
  3. test(gateway): add interaction controls test suite (62 tests) — full test coverage including backward compat stubs

Testing

  • 62 new interaction tests covering clarify flows, approval flows, timeout handling, callback routing, and Telegram-specific rendering
  • 9 existing test files updated with _pending_clarifications stubs to maintain backward compatibility
  • All tests pass on latest main (rebased)

Files changed (20 files, ~1850 additions)

Area Files Risk
Controls model base.py, whatsapp.py Low
Lifecycle + Telegram run.py, telegram.py, clarify_tool.py, send_message_tool.py Medium
Tests 14 test files (2 new, 12 updated) Low

@gaelcado
gaelcado force-pushed the pr/telegram-inline-support branch from 2a0fc97 to 5404f5b Compare March 20, 2026 20:00
@gaelcado gaelcado changed the title feat(gateway): generic interaction controls with Telegram as first native renderer feat(gateway): interactive controls with Telegram inline keyboards (Phase 1 of #503) Mar 20, 2026
@gaelcado

gaelcado commented Mar 20, 2026

Copy link
Copy Markdown
Author

Rebased this onto latest main and reorganized into 3 clean, incrementally reviewable commits. Quick summary of where this stands:

Status: Rebased, all 5719 tests pass (the 3 failures are pre-existing upstream issues in transcription/delegate tests). I'll keep this rebased every 2-3 days.

Why this is low-risk:

  • Purely additive — zero breaking changes to existing platform behavior
  • Every platform without native support gets automatic text fallback via render_controls_as_text()
  • 62 dedicated tests + 9 existing test stubs updated for backward compat

Design alignment with #503:

  • Transport-agnostic ControlsDef/ButtonDef model that Discord, Slack, and WhatsApp adapters can adopt later by overriding a single send_interactive() method
  • Telegram is just the first native renderer as a proof of concept

Let me know if anything needs changing!

@dlkakbs

dlkakbs commented Mar 23, 2026

Copy link
Copy Markdown
Contributor

Would it make sense to also use inline keyboards for long message truncation? When a response exceeds 25 lines, show the first 25 with an 'Expand 📄' button.

gaelcado and others added 4 commits March 24, 2026 16:54
…model

Introduce ControlsDef and ButtonDef TypedDicts as a transport-agnostic
abstraction for interactive controls (buttons, option lists). Add
render_controls_as_text() fallback for platforms without native support,
MAX_INTERACTIVE_BUTTONS on BasePlatformAdapter, and send_interactive()
base method with automatic text fallback.

This serves as the shared component model for gateway and future RPC,
as discussed in #503.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ram inline keyboards

Wire ControlsDef into the clarify and approval flows:
- Clarify tool gains controls parameter with 120s timeout
- Approval uses inline accept/reject buttons
- _pending_clarifications dict tracks active sessions
- Telegram adapter builds native InlineKeyboardMarkup via
  build_telegram_reply_markup() and handles CallbackQueryHandler
- send_interactive() Telegram override for rich button rendering
- Cleanup on /stop and shutdown

Phase 1 of #503.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add comprehensive test coverage for the interactive controls feature:
- test_approval_interactions.py (new, 326 lines)
- test_clarify_interactions.py (new, 389 lines)
- test_telegram_conflict.py (updated)
- test_telegram_text_batching.py (updated)
- test_send_message_tool.py (updated)
- 9 existing test files updated with _pending_clarifications stubs

All 62 interaction tests pass. Backward compatibility verified by
stub updates across existing gateway test suite.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@gaelcado
gaelcado force-pushed the pr/telegram-inline-support branch from 5404f5b to beeb8fb Compare March 24, 2026 17:43
@gaelcado

Copy link
Copy Markdown
Author

(Rebased on latest main; resolved one gateway/run.py conflict with no scope change)


Would it make sense to also use inline keyboards for long message truncation? When a response exceeds 25 lines, show the first 25 with an 'Expand 📄' button.

For this PR, I kept scope strictly to Issue #503 (transport-agnostic controls + Telegram clarify/approval lifecycle) to keep changes focused and incremental.

I’d prefer to land this first, then open a small follow-up PR for long-message expand/collapse

Feel free to open an issue!

@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/gateway Gateway runner, session dispatch, delivery platform/telegram Telegram bot adapter labels May 3, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Thanks for this @gaelcado — Phase 1 of #503 is one of the more thoughtful design proposals we've gotten, and the transport-agnostic ControlsDef/ButtonDef shape directly informed how we built this out.

Closing because the feature has since shipped on main:

  • feat(gateway): wire clarify tool with inline keyboard buttons on Telegram #24199 (May 12) — feat(gateway): wire clarify tool with inline keyboard buttons on Telegram. Introduces tools/clarify_gateway.py as a shared event-based primitive and an abstract send_clarify on BasePlatformAdapter with a numbered-text fallback, so every adapter gets a working clarify out of the box and Telegram renders one inline button per choice plus "Other (type answer)."
  • fix(discord): render clarify choices as buttons #19249 salvage (May 13) — feat(discord): render clarify choices as buttons. Brings Discord to parity by overriding send_clarify with ClarifyChoiceView.
  • a83d579fix(telegram): enforce gateway auth for inline approval callbacks. Inline-keyboard approvals for dangerous commands are also live and being security-hardened.

The merged design follows the same shape you proposed (transport-agnostic core + per-platform native renderers + numbered-text fallback), so credit for the direction is yours. We won't be cherry-picking from this branch — it's ~5700 commits behind main and the merged feature now lives in the same files, so a salvage would be a near-total rewrite.

If you want to tackle Phase 2 (execution plans / structured todo presentation from #503) on top of the current send_clarify / clarify_gateway scaffolding, that'd be a great follow-up.

@teknium1 teknium1 closed this May 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P3 Low — cosmetic, nice to have platform/telegram Telegram bot adapter type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants