test(integration): expand MCP-functional tier (Phase 1 of test ramp-up) - #136
Merged
Merged
Conversation
… multi-tool turns, error paths Continuation of the test infrastructure ramp-up after #131. The new MCP functional tier shipped with 4 cases (end_turn + send happy paths). This expands it to 11 — broader coverage on the messaging tool surface, multi-tool turns, error paths, and chat-id routing. Changes: 1. New `recording-handler.ts` factored out of `talon-mcp-functional.test.ts`. The previous inline `makeRecordingHandler` is now a shared helper with: - `byAction(name)` — ergonomic filter by action name - `last()` — most recent capture - `respond` map — per-action response overrides for error injection - Synthetic responses for the full messaging surface (send_message, send_photo, react, edit_message, delete_message, etc.) plus stop_poll, send_chat_action, and read paths returning `{ok: true, items: []}` - `seq` field on captures — preserves emit order for ordering assertions 2. `talon-mcp-functional.test.ts` — refactored to use the shared helper and added 7 new cases: Broader tool surface (5 cases): - `react` tool_use → handler captures `react` action with message_id + emoji - `edit_message` tool_use → captures with new text - `delete_message` tool_use → captures with message_id - `pin_message` tool_use → captures with message_id Multi-tool turn (1 case): - `react` + `send` + `end_turn` in one assistant message → all three captured in the order the model emitted them. Verifies sequencing is preserved through the SDK→MCP→bridge chain. Confirms `end_turn()` with empty input doesn't post. Error paths (1 case): - Recording handler responds `{ok: false, error: ...}` to send_message. The handler still gets called, the tool_use is observed, the SDK loop completes cleanly (no crash). Documents that ok-false responses are part of the normal flow, not exceptions. Chat ID routing (1 case): - String chatId from runTalonTurn → gateway hashes to a synthetic numeric id → consistent across all captures within a turn → recording handler sees the same numeric chatId on every action. Verifies `findContextByStringId` routing. 3. Per Dylan's "this is just the beginning of the ramp up, work on it during heartbeats" — leaves obvious next steps: - Multi-turn dialogues (script several turns, exercise session resume) - History tool dispatch (read_chat_history, search_chat_history) - Member/admin tool dispatch - Sticker tool dispatch (set creation, add/delete) - Streaming behaviors (text deltas, thinking deltas observable in tests) - Flow violation paths (trailing prose without end_turn → reprompt) - Cross-tool dedup (model calls send + end_turn with same text) - Plugin MCP servers (extras-tools, wikipedia-tools, etc.) 11 / 11 new MCP functional tests pass. 20 / 32 integration tests pass (12 skipped — the live-Telegram suites gated on `TALON_TEST_CHAT_ID`). tsc + prettier + oxlint clean.
This was referenced May 10, 2026
Merged
dylanneve1
pushed a commit
that referenced
this pull request
May 10, 2026
… streaming deltas Batch 2 of the MCP-functional test expansion queue from PR #136. 4 new describe suites, 11 new test cases: **Multi-turn session state** (2 tests) - Verifies `turns` counter increments across two sequential `runTalonTurn` calls on the same chatId without session reset. Catches any accidental `resetSession` inside `handleMessage`. - Verifies `resetSession: true` actually clears state — seeds session.turns=99, confirms it's 1 after a fresh turn. **History and info tool dispatch** (3 tests) - `read_chat_history` routes through full SDK→MCP→bridge→gateway chain and recording handler captures the action body + limit param. - `get_chat_info` routes correctly (confirms it's in mcp-server.ts registry). - Multi-read turn: `read_chat_history + get_chat_info + end_turn` in one assistant message — all three captured in dispatch order. **Streaming text deltas** (2 tests) - Text block emitted before `end_turn` appears in `streamDeltas` (proves `onStreamDelta` callback is wired through the real handler and SDK). - Pure text turn (no tool_use): `streamDeltas` captured, recording handler gets nothing (no MCP dispatch). **Additional messaging tool dispatch** (4 tests) - `unpin_message` — ok-only action with message_id. - `stop_poll` — ok-only action with message_id. - `forward_message` — ok-only action with source_chat_id + message_id. - `list_pinned` — read-path action, recording handler returns items array. Each suite gets its own `RecordingHandler` with a distinct `seed` (2000–5000) so synthetic message_ids don't collide if suites ever run concurrently. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
claudiusthebot
added a commit
that referenced
this pull request
May 10, 2026
…ools, streaming deltas (#138) * test(integration): MCP-functional tier 2 — multi-turn, history tools, streaming deltas Batch 2 of the MCP-functional test expansion queue from PR #136. 4 new describe suites, 11 new test cases: **Multi-turn session state** (2 tests) - Verifies `turns` counter increments across two sequential `runTalonTurn` calls on the same chatId without session reset. Catches any accidental `resetSession` inside `handleMessage`. - Verifies `resetSession: true` actually clears state — seeds session.turns=99, confirms it's 1 after a fresh turn. **History and info tool dispatch** (3 tests) - `read_chat_history` routes through full SDK→MCP→bridge→gateway chain and recording handler captures the action body + limit param. - `get_chat_info` routes correctly (confirms it's in mcp-server.ts registry). - Multi-read turn: `read_chat_history + get_chat_info + end_turn` in one assistant message — all three captured in dispatch order. **Streaming text deltas** (2 tests) - Text block emitted before `end_turn` appears in `streamDeltas` (proves `onStreamDelta` callback is wired through the real handler and SDK). - Pure text turn (no tool_use): `streamDeltas` captured, recording handler gets nothing (no MCP dispatch). **Additional messaging tool dispatch** (4 tests) - `unpin_message` — ok-only action with message_id. - `stop_poll` — ok-only action with message_id. - `forward_message` — ok-only action with source_chat_id + message_id. - `list_pinned` — read-path action, recording handler returns items array. Each suite gets its own `RecordingHandler` with a distinct `seed` (2000–5000) so synthetic message_ids don't collide if suites ever run concurrently. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(test): correct 6 MCP-functional-tier-2 test failures Diagnosed from CI log (hb #202, 2026-05-10): 1. read_chat_history bridge name: tool calls bridge("read_history", ...) not bridge("read_chat_history", ...). Assertions updated accordingly. Also added "read_history" case to recording-handler for correct response. 2. Multi-read test: same fix — byAction("read_chat_history") → byAction("read_history") in 2 places. 3+4. Streaming delta tests: the stub binary emits complete content blocks, not streaming delta events. onStreamDelta never fires. Changed test #3 to assert on send_message dispatch instead. Changed test #4 to assert "no MCP dispatch, no tool calls" (the meaningful invariant for a prose-only turn that triggers the flow-violation handler). 5. forward_message: tool schema only exposes message_id. source_chat_id is stripped by Zod. Changed input + assertion to only use message_id. 6. list_pinned: no production tool routes to bridge("list_pinned", ...). Replaced with get_message_by_id which exists in history.ts and the recording handler returns ok:true items:[] for it. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Continuation of the test infrastructure ramp-up Dylan asked for after #131 merged. The new MCP functional tier shipped with 4 cases (end_turn + send happy paths). This expands it to 11 — broader coverage on the messaging tool surface, multi-tool turns, error paths, and chat-id routing.
Changes
recording-handler.ts(new shared helper)Factored out of
talon-mcp-functional.test.ts. Now a reusable helper:byAction(name)— ergonomic filterlast()— most recent capturerespondmap — per-action response overrides for error injectionseqfield on captures — preserves emit order for ordering assertionsFuture MCP-tier suites can drop this in without re-implementing.
talon-mcp-functional.test.ts(+7 cases)Broader tool surface (5 cases):
react→ capturesreactaction withmessage_id+emojiedit_message→ captures with new textdelete_message→ captures withmessage_idpin_message→ captures withmessage_idMulti-tool turn (1 case):
react+send+end_turnall in one assistant message. Verifies sequencing preserved through SDK→MCP→bridge chain. Confirmsend_turn()with empty input doesn't post.Error path (1 case):
{ok: false, error: "..."}tosend_message. Handler still gets called, tool_use observed, SDK loop completes cleanly. Documents ok-false responses are normal flow, not exceptions.Chat ID routing (1 case):
findContextByStringIdrouting.Why this matters
Each new case asserts on a wire-shape contract that nothing else in the test suite covers. If any of these regress, the failure points to a specific tool's
execute(params, bridge)mapping, the action handler's switch, or the gateway's routing.What's next (heartbeat work)
Per Dylan's "this is just the beginning of the ramp up, work on it during heartbeats" — obvious follow-ups:
read_chat_history,search_chat_history,get_message_by_id)end_turn→ reprompt)send+end_turnwith same text)extras-tools,wikipedia-tools, etc.)Test plan
tsc --noEmitcleanprettier --checkclean on touched filesoxlintclean🤖 Generated with Claude Code