feat(tools): MCP sampling Phase 2 — spec coverage, governance & audit

[eren-karakus0]

Summary

Extends the MCP sampling/createMessage implementation with full MCP spec compliance, per-server governance controls, and structured audit observability. Built on top of Phase 1 (PR #64).

Spec Coverage

• Bug fix: _map_stop_reason now correctly maps tool_calls → "toolUse" (was incorrectly "endTurn")
• tools/toolChoice forwarding: Server-provided tool definitions are forwarded to the LLM as OpenAI function-calling format, with toolChoice mode mapping (auto/required/none)
• Tool content blocks: Full support for tool_use and tool_result content blocks in _convert_sampling_messages — converts to OpenAI tool_calls and role: "tool" messages respectively
• Tool use responses: When LLM responds with tool_calls, returns CreateMessageResult with stopReason: "toolUse" and tool use content blocks
• Malformed args safety: json.loads() on tool call arguments is now wrapped in try/except — malformed JSON falls back to raw string instead of crashing the callback
• Mixed content lists: Content lists containing both text and tool_result blocks are now handled correctly (previously tool_results were silently dropped as "unsupported")

Governance

• allowed_models: Per-server model whitelist. Empty list (default) permits any model. Non-listed models return structured ErrorData
• max_tool_rounds: Caps consecutive tool use rounds per server (default: 5). Prevents infinite tool loops between LLM and server. Set to 0 to disable tool loops entirely. Counter resets on normal text response

Audit Observability

• _sampling_metrics: Per-server counters tracking requests, errors, tokens_used, and tool_use_count
• get_mcp_status(): Metrics exposed in server status for monitoring/debugging
• log_level: Config-driven audit verbosity ("debug", "info", "warning")

Config Schema (all backward-compatible)

mcp_servers:
  my_server:
    sampling:
      # Existing (unchanged):
      enabled: true
      model: "gemini-3-flash"
      max_tokens_cap: 4096
      timeout: 30
      max_rpm: 10
      # New:
      allowed_models: []        # model whitelist (empty = all)
      max_tool_rounds: 5        # tool loop limit (0 = disable)
      log_level: "info"         # audit verbosity

All new fields have defaults — zero breaking changes for existing configurations.

Files Changed

File	Change	Delta
tools/mcp_tool.py	Bug fix, tool content conversion, tools/toolChoice forwarding, governance, metrics	+170 lines
tests/tools/test_mcp_tool.py	31 new tests across 7 test classes	+560 lines
docs/mcp.md	Config table, Tool Use section, Per-Server Policy section	+90 lines
skills/mcp/native-mcp/SKILL.md	Config table, tool use reference	+15 lines

Test Plan

• pytest tests/tools/test_mcp_tool.py -q → 151 passed, 3 pre-existing fail (mcp pkg not installed)
• pytest tests/tools/test_mcp_tool.py -k Sampling -W error::RuntimeWarning → 0 warnings
• All 31 new tests pass: stopReason fix, tool content conversion (6), tools/toolChoice forwarding (6), audit metrics (4), allowed_models (3), max_tool_rounds (4), log_level (1), malformed args (3), mixed content list (3), existing test updated (1)
• Backward compatibility: existing 120 tests unaffected
• Tested on Windows 11 with Python 3.13.5

Related

• Built on Phase 1 sampling implementation (PR feat(mcp): native MCP client with full spec support #64)
• References MCP spec 2025-11-25 sampling/createMessage

[teknium1]

Thanks for this work! The sampling concept and the thorough documentation were valuable. We've incorporated the feature in PR #753, restructured as a class-based SamplingHandler design with some bug fixes (the tool use path needed CreateMessageResultWithTools + ToolUseContent instead of CreateMessageResult with raw dicts, and SamplingCapability needed to advertise tool support). You're credited in the commit message.

Closing this PR since the branch was 388 commits behind main and couldn't be merged directly, but the feature is landing. Thanks again for the contribution!

[eren-karakus0]

Glad to see MCP sampling landed in #753! Looks like it covers similar ground to what I proposed here. Happy to have contributed to the direction.

[SafariShow]

Hermes Agent Review

Found 1 critical issue, 1 warning, and a few minor suggestions.

🔴 Critical

• apps/dashboard/server/api/ai/personalize-template.post.ts and apps/dashboard/server/api/ai/template-events.post.ts: Missing authentication checks. The comment says "Auth: requires the same Bearer/cookie token", but neither endpoint actually verifies the user. An attacker can hit personalize-template to drain Anthropic API tokens via Claude generation, and hit template-events to spam the builder_events table with junk. You need to call getTemplateAccessContext or similar auth middleware before processing the body.

⚠️ Warnings

• apps/dashboard/server/utils/templateEvents.ts (Lines 76-88): emitTemplateEvent triggers a db.insert() without returning a Promise, and it is not passed to event.waitUntil(). In serverless environments (like Vercel), if the HTTP response is returned before the background DB insert completes, the process will be suspended and the event will be dropped. Either pass the H3Event to use event.waitUntil(), or simply await it with a .catch() in the endpoint.
• Request Body Validation: readBody(event) as RequestBody is used without Zod parsing. If body.products is passed as a string or object instead of an array, the application will crash at runtime when calling .map() or .length.

💡 Suggestions

• apps/dashboard/server/ai/template-personalizers/listicle.ts: The file header comment says "Per-item copy rewrite is intentionally deferred to Phase 2", but the code actually does implement rewriteListicleItemsWithLLM! The docstring is stale and should be updated to match the Phase 3 implementation.
• apps/dashboard/scripts/seed-global-templates.ts: Contains several console.log statements for progress reporting. Perfectly fine for a CLI script, but noting it here in case you intended to use a structured logger.

✅ Looks Good

• The architectural split between the generic substitution (Floor) and the LLM rewrite (Ceiling) is very clean and provides a great fallback mechanism.
• Great job adding the Authorization: Bearer fallback in resolveAuthToken to support SDK/MCP callers!
• Zod schemas in llm-copy.ts for structured outputs are well-described and strict.