Skip to content

[Z2O-1694] fix(mcp): resolve ${context:} headers caller-side, not on the MCP loop - #2

Merged
chungty merged 3 commits into
mainfrom
z2o-1694-mcp-context-headers-caller-resolve
Jun 2, 2026
Merged

[Z2O-1694] fix(mcp): resolve ${context:} headers caller-side, not on the MCP loop#2
chungty merged 3 commits into
mainfrom
z2o-1694-mcp-context-headers-caller-resolve

Conversation

@chungty

@chungty chungty commented Jun 2, 2026

Copy link
Copy Markdown

Why

${context:NAME} templated MCP headers — the mechanism delegated-principal propagation relies on — silently no-op for every streamable-HTTP MCP server (the default for remote servers).

The request event hook (_inject_templated_headers) resolves ${context:} by reading get_session_env(...) — i.e. the ContextVar in whatever task drives the HTTP POST. But the POST runs on the dedicated MCP background loop, in a task the SDK's post_writer spawns under a task group created at server-connect time. That task's contextvar context is frozen at startup and never sees the per-turn session vars set on the caller's thread. So resolution always returns "" and the header is dropped.

Diagnosis (end-to-end, from Mercator's delegated-principal feature)

  • Mercator's plugin resolved the principal correctly — logs: delegated_principal_resolution status=resolved ... @verdigris.co.
  • Yet Meridian's whoami returned email: null.
  • A direct curl to the Meridian gateway isolated it — gateway honors the header, Hermes was sending it empty:
# WITH header    -> {"email":"thomas@verdigris.co", ...}
# WITHOUT header -> {"email":null, ...}    # == what Hermes actually sent

The hook's own docstring claimed it resolved "against the calling task's session context" — an invariant that does not hold for streamable HTTP, because post_writer decouples the request from the caller via write_stream (and it's a different event loop entirely).

Fix

Resolve the templates where the context is correct — the caller's thread (the synchronous tool handler) — and bridge the resolved values to the request hook via an instance attribute:

  1. MCPServerTask._resolve_templated_headers() — resolves ${context:} against the current (caller) task's context, returning only non-empty values. Called from the sync tool handler.
  2. The handler stashes the result on server._outbound_resolved_headers inside _rpc_lock (so concurrent per-server calls can't race) and clears it in a finally (so pings / reconnect GETs never carry a stale principal).
  3. The request hook now applies self._outbound_resolved_headers via the new _apply_resolved_headers helper instead of resolving against its own (wrong) context. The cross-origin identity-header strip is preserved.

SSE / legacy-HTTP transports already freeze templated headers at connect (a documented limitation) and are unchanged.

Why caller-side is correct

The sync tool handler runs on the agent thread where plugin hooks (pre_tool_call / pre_gateway_dispatch) set the session vars; _rpc_lock serializes calls per server, so a single "current outbound headers" slot on the instance is race-free.

Tests

tests/tools/test_mcp_context_template.py — existing helper + httpx-mechanics tests retained (annotated); new coverage:

  • TestResolveTemplatedHeadersMethod — caller-side resolution (set/unset/none).
  • TestApplyResolvedHeaders — loop-side application: inject same-origin, drop when missing, strip cross-origin.
  • test_resolved_headers_survive_lost_caller_context — the crux regression: a value resolved while the var is set still injects after that context is gone (simulating the MCP loop). This fails on main.

Run: 64 passed across the context-template + session-env suites; mcp circuit-breaker / error / url / cancellation suites green (29 passed, 1 skipped).

Rollout

After merge, Mercator bumps its Hermes pin to this SHA; @Mercator whoami should then return the caller's email with no Mercator-side change (Mercator PR NousResearch#30 already resolves the principal at the request root). Tracked: Z2O-1694; relates to MER-62, Z2O-1691.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Tests

    • Added unit and integration tests confirming per-call resolution and application of templated MCP headers, cross-origin identity stripping, persistence of resolved values across async boundaries, and that resolved headers are cleared between calls.
  • Improvements

    • Resolve context-based header templates on the caller side and apply stored results to outbound requests, omitting or removing identity headers when empty or cross-origin for more reliable request behavior.

…the MCP loop

${context:NAME} templated MCP headers (delegated principal, signed
assertions) were resolved inside the httpx request event hook — which
runs on the dedicated MCP background loop, in a task spawned by the
SDK's post_writer. That task's contextvar context is frozen at
server-connect time and never sees the per-turn session vars set on the
caller's thread, so resolution always returned "" and the header was
dropped. Net effect: delegated-principal propagation silently no-ops for
every streamable-HTTP MCP server (the default for remote servers).

Diagnosed end-to-end from Mercator: the plugin resolved the principal
(logs: status=resolved @verdigris.co) but Meridian saw email:null; a
direct curl to the gateway with vs without the header confirmed the
gateway honors it and Hermes was sending it empty.

Fix: resolve the templates on the CALLER's thread (the sync tool
handler, where the session vars are live), bridge the resolved values
onto the server instance, and have the request hook apply them. The
apply happens inside _rpc_lock so concurrent per-server calls can't
race, and is cleared after each call so non-tool requests (pings,
reconnect GETs) never carry a stale principal.

- _resolve_context_templates stays caller-side via the new
  MCPServerTask._resolve_templated_headers().
- _apply_resolved_headers (module helper) stamps pre-resolved values,
  preserving the cross-origin identity-header strip.
- The request hook reads self._outbound_resolved_headers instead of
  resolving against its own (wrong) context.

SSE / legacy-HTTP transports already freeze templated headers at
connect (documented limitation) and are unchanged.

Tests: caller-side resolution, loop-side application, cross-origin
strip, and the crux regression — a value resolved while the var is set
still injects after that context is gone (simulating the MCP loop).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@linear-code

linear-code Bot commented Jun 2, 2026

Copy link
Copy Markdown

Z2O-1694

MER-62

Z2O-1691

@coderabbitai

coderabbitai Bot commented Jun 2, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 5dbdf9ca-9cb8-42a2-a583-24c54c76439d

📥 Commits

Reviewing files that changed from the base of the PR and between e9aec14 and d31a0b1.

📒 Files selected for processing (1)
  • tests/tools/test_mcp_structured_content.py

📝 Walkthrough

Walkthrough

The PR moves ${context:NAME} MCP header template resolution from the MCP loop into the caller thread, stores templates at connect time, resolves per-call values before tool invocation, bridges them into MCPServerTask._outbound_resolved_headers for the duration of each RPC, and applies them in the httpx request hook with cross-origin identity stripping.

Changes

Caller-thread header resolution with loop-side application

Layer / File(s) Summary
Tests: imports, resolution, application, scoping
tests/tools/test_mcp_context_template.py, tests/tools/test_mcp_structured_content.py
Import MCPServerTask and _apply_resolved_headers; add unit tests for _resolve_templated_headers() and _apply_resolved_headers(), contract test for persisted resolved values, RPC scoping test, update httpx hook comment, and bind _rpc onto fake server fixture.
Storage slots, imports, and header-application helper
tools/mcp_tool.py
Add contextlib import; extend MCPServerTask.__slots__ with _templated_headers and _outbound_resolved_headers; add _apply_resolved_headers(request, *, same_origin: bool, names, resolved) to stamp resolved headers, remove headers with empty values, and strip identity headers on cross-origin.
Resolve templates and RPC-scoped bridging
tools/mcp_tool.py
Add MCPServerTask._resolve_templated_headers() to resolve stored ${context:NAME} templates against caller session context and update MCPServerTask._rpc(resolved_headers=...) to set _outbound_resolved_headers under _rpc_lock and clear it in finally; run keepalive inside _rpc().
Request hook application and handler bridging
tools/mcp_tool.py
Modify httpx event_hooks["request"] to apply _outbound_resolved_headers via _apply_resolved_headers() (preserving same-origin vs cross-origin behavior); stash templated headers at connect time; update _make_tool_handler to call server._resolve_templated_headers() on caller thread and pass mapping into server._rpc(...).
Utility handlers use caller-side resolution
tools/mcp_tool.py
Update list_resources, read_resource, list_prompts, and get_prompt to resolve per-call templated headers on the caller thread and pass them into server._rpc(...).

Sequence Diagram

sequenceDiagram
  actor Caller
  participant MCPServerTask
  participant httpxRequestHook
  participant Remote

  Caller->>MCPServerTask: _resolve_templated_headers()
  MCPServerTask->>MCPServerTask: set _outbound_resolved_headers (under _rpc_lock)
  Caller->>MCPServerTask: enter _rpc(...) context with bridged headers
  httpxRequestHook->>MCPServerTask: read _outbound_resolved_headers
  httpxRequestHook->>httpxRequestHook: _apply_resolved_headers(request, same_origin?, names, resolved)
  httpxRequestHook->>Remote: send outbound HTTP request with applied headers
  MCPServerTask->>MCPServerTask: clear _outbound_resolved_headers in finally
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

  • VerdigrisAI/hermes-agent#1: Related work introducing session-context APIs/tests used for per-request ${context:NAME} resolution.

Poem

🐰 In caller's burrow templates wake,

Resolved and bridged for the loop to take,
Headers hop on when origins align,
Stripped or dropped when borders cross the line,
Cleared at end — no leaks for this small hare.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and specifically summarizes the main change: resolving ${context:} MCP headers on the caller side instead of the MCP loop, which is the primary fix described in the PR objectives.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch z2o-1694-mcp-context-headers-caller-resolve

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions

github-actions Bot commented Jun 2, 2026

Copy link
Copy Markdown

🔎 Lint report: z2o-1694-mcp-context-headers-caller-resolve vs origin/main

ruff

Total: 0 on HEAD, 0 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 0 pre-existing issues carried over.

ty (type checker)

Total: 9077 on HEAD, 9077 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 4805 pre-existing issues carried over.

Diagnostics are surfaced as warnings — this check never fails the build.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request addresses issue Z2O-1694 by resolving templated headers on the caller's thread where the session context is active, and bridging these values to the MCP background loop via the MCPServerTask._outbound_resolved_headers instance attribute under _rpc_lock. Feedback on these changes highlights a potential security risk where background keepalive pings could leak stashed user headers, and notes that other handlers (such as resource and prompt handlers) currently omit this header resolution. Additionally, a minor code simplification was suggested to use pop instead of del when removing empty headers in _apply_resolved_headers.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread tools/mcp_tool.py Outdated
Comment thread tools/mcp_tool.py Outdated
chungty and others added 2 commits June 1, 2026 23:30
…alive principal leak

Gemini review on PR #2 flagged two real issues:

1. (high) Other MCP handlers (list_resources, read_resource, list_prompts,
   get_prompt) also issue outbound HTTP but didn't resolve/stash templated
   headers — delegated principal silently dropped for resource/prompt ops.
2. (high) Background keepalive ping (list_tools in _wait_for_lifecycle_event)
   doesn't take _rpc_lock, so it could overlap a user tool call and pick up
   the stashed _outbound_resolved_headers — leaking the user's principal on a
   system keepalive.

Fix:
- Extract `MCPServerTask._rpc(resolved_headers=None)` async context manager:
  acquires _rpc_lock, exposes the caller-resolved headers for the call's
  duration, clears them after. System RPCs pass no headers and still hold the
  lock, so they can't overlap-and-read a user call's principal.
- Use it in all five user-facing handlers (tool + 4 resource/prompt) with
  caller-side _resolve_templated_headers(), and in the keepalive ping (empty
  headers) — closing the leak.
- (medium) Simplify _apply_resolved_headers drop path to headers.pop(name, None).

Header partitioning (static vs templated, Gemini ref #3) was already in place
via _split_static_and_templated_headers — only templated names are resolved.

Tests: + _rpc scoping/clear contract (system RPC carries no headers).
94 passed across context-template + mcp + session-env suites.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…eader resolution

test_mcp_structured_content fakes `server` as a SimpleNamespace; the tool
handler now calls server._resolve_templated_headers() (caller-side) and
server._rpc() under the lock, which the bare fake lacked → AttributeError.
Give the fake an empty resolver and bind the real _rpc CM (it only needs
_rpc_lock + _outbound_resolved_headers, both present). Pure test fixture fix;
no production change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@chungty
chungty merged commit f532c6b into main Jun 2, 2026
16 checks passed
@chungty
chungty deleted the z2o-1694-mcp-context-headers-caller-resolve branch June 2, 2026 06:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant