Skip to content

fix(anthropic): alias session_search/memory OAuth billing-classifier triggers (fixes #65365, #82154) - #78025

Open
JoaoMarcos44 wants to merge 1 commit into
NousResearch:mainfrom
JoaoMarcos44:fix/65365-oauth-session-search-memory-alias
Open

fix(anthropic): alias session_search/memory OAuth billing-classifier triggers (fixes #65365, #82154)#78025
JoaoMarcos44 wants to merge 1 commit into
NousResearch:mainfrom
JoaoMarcos44:fix/65365-oauth-session-search-memory-alias

Conversation

@JoaoMarcos44

@JoaoMarcos44 JoaoMarcos44 commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Fixes #65365

Summary

On Anthropic subscription OAuth (claude_code credential), Hermes sessions carrying the session_search or memory toolset are misrouted by Anthropic's OAuth billing classifier into the (usually empty) extra-usage lane, surfacing as HTTP 400 "You're out of extra usage" on a valid subscription — or silently billing the metered lane on accounts where extra usage is enabled.

Related to #76807 (open) — that PR aliases the session_search/memory schema names, descriptions, and prose. This PR implements the same class of fix independently, and additionally closes a gap the repo's own triage bot flagged in #76807: named tool_choice is not routed through the alias, so a forced tool_choice="session_search" would (a) still leak the literal trigger string onto the wire and (b) reference a tool name that no longer matches any entry in tools[] once the schema itself is aliased — a guaranteed 400 on top of the billing misroute. Happy to consolidate with #76807 if the maintainers prefer — opening this as a complete, independently-tested alternative so there's a mergeable option either way.

Root cause (live-verified, not speculative)

Per the issue thread's own deterministic A/B repros (replayed byte-exact request bodies against /v1/messages with the real OAuth client, read via anthropic-ratelimit-unified-representative-claim / -overage-in-use response headers as a lane oracle — no dependency on the laggy usage counter):

  • Tool schemas are innocent — a full 44-tool request with session_search/memory schemas succeeds when the system prompt is minimal.
  • The trigger is specific system-prompt prose: three sentences, jointly required — the SESSION_SEARCH_GUIDANCE recall sentence, and two SKILLS_GUIDANCE sentences ("save the approach as a skill with skill_manage" / "patch it immediately with skill_manage(action='patch')"). Breaking any one of the three clears the classifier.
  • This has been stable prose since March 2026 (1ecfe6867) — Anthropic's classifier changed, not Hermes. Per one reporter: "any fix here is chasing a moving server-side target, and other prose could start tripping it next."

Fix — two independent layers

1. OAuth wire alias (agent/anthropic_adapter.py, agent/transports/anthropic.py) — breaks the session_search leg:

  • session_searchchat_history_lookup, memorycontext_notes on the OAuth wire only (tool name + description).
  • session_search also aliased in system-prompt prose via word-boundary regex (memory is deliberately not prose-aliased — it's ordinary English throughout the prompt and inside the memory tool's own parameter docs; rewriting it there would corrupt guidance the model must follow verbatim).
  • Collision guard: if a real tool already owns the alias's wire name, the alias is skipped for that request rather than emitting two identical tool names (a guaranteed 400, strictly worse than the bug being fixed).
  • tool_choice gap (new vs. fix(anthropic): alias the session_search and memory schemas on the OAuth wire #76807): a forced tool choice is now routed through the same normalizer as tools[], so it always matches the corresponding entry instead of leaking the raw trigger string or a stale name.
  • normalize_response reverse-maps the alias last, after the existing [Bug]: Anthropic OAuth strips mcp_ prefix from Hermes-native MCP tool names, breaking registry lookup #25255 registry lookups, so a genuinely registered tool under that wire name always wins.
  • OAuth-only: API-key requests are byte-identical to before this change.

2. Prompt-preserving reword (agent/prompt_builder.py) — breaks the skill_manage leg, as defense-in-depth against the classifier's demonstrated instability:

  • Rewords the two triggering SKILLS_GUIDANCE sentences while keeping the same meaning, still naming skill_manage, and leaving the ## Skill Safety Rule section byte-identical (existing test_ghost_skill_pruning.py assertions cover that section and still pass unmodified).
  • Applies to all auth paths (API key and OAuth) — it's a prompt-copy improvement, not a wire-level transform.

Two independent layers rather than one because the three-sentence AND-condition means a future classifier tightening could start firing on either remaining leg alone; breaking both removes the fragility instead of chasing the classifier one leg at a time.

Test plan

  • 15 new tests in tests/agent/test_anthropic_mcp_prefix_strip.py: name/description/prose aliasing, word-boundary safety (session_search_tool.py path survives), tool_choice alias + non-aliased-name prefix, wire-collision avoidance, API-key passthrough, response-side round-trip (both directions), registered-tool-wins precedence
  • 2 new tests in tests/agent/test_ghost_skill_pruning.py: trigger phrasing removed, skill_manage still named, Skill Safety Rule untouched
  • tests/agent/ full suite: no regressions
  • tests/agent/transports/: 228 passed, no regressions
%%{init: {'theme': 'dark', 'themeVariables': { 'primaryColor': '#00f0ff', 'mainBkg': '#0a0a16', 'primaryTextColor': '#ffffff', 'primaryBorderColor': '#ff007f', 'lineColor': '#00f0ff'}}}%%
graph TD
    A[🔒 OAuth Request Built] -->|3-Sentence AND-Trigger| B{Classifier Fingerprint?}
    B -->|session_search prose + name| C[⚡ Wire Alias: chat_history_lookup]
    B -->|skill_manage x2 prose| D[⚡ Prompt Reword: Layer 2]
    C --> E[🧬 tool_choice Routed Through Same Alias]
    D --> F[🛰️ Skill Safety Rule Untouched]
    E --> G[🚀 Subscription Lane: five_hour]
    F --> G
Loading

Infographic :

infographic

…triggers

Anthropic subscription OAuth (claude_code credential) misroutes Hermes
sessions carrying the session_search or memory toolset into the
extra-usage lane, surfacing as HTTP 400 "You're out of extra usage" on
a valid subscription. Live-verified via the
anthropic-ratelimit-unified-representative-claim response header
(deterministic lane oracle, no dependency on the laggy usage counter):
tool schemas are innocent, the trigger is three specific system-prompt
sentences (session_search recall + two skill_manage sentences),
required jointly — breaking any one clears the classifier.

Two independent layers:
1. OAuth wire alias (anthropic_adapter.py, transports/anthropic.py):
   session_search -> chat_history_lookup, memory -> context_notes in
   tool name, description, and (session_search only) system-prompt
   prose, with wire-collision guarding and a normalize_response
   reverse-map that keeps NousResearchGH-25255 registered-tool precedence. Also
   routes named tool_choice through the same normalizer, closing a gap
   where a forced tool_choice would leak the raw trigger string and
   stop matching tools[].
2. Prompt-preserving reword (prompt_builder.py): rewords the two
   triggering SKILLS_GUIDANCE sentences while keeping the same
   meaning, still naming skill_manage, and leaving the Skill Safety
   Rule section untouched. Applies to all auth paths since it's a
   prompt-copy change, not a wire-level transform.

Two layers rather than one because the three-sentence AND-condition
means a classifier tightening could start firing on either remaining
leg alone.

Fixes NousResearch#65365
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/anthropic Anthropic native Messages API P2 Medium — degraded but workaround exists needs-decision Awaiting maintainer decision before any implementation labels Aug 3, 2026
@JoaoMarcos44 JoaoMarcos44 changed the title fix(anthropic): alias session_search/memory OAuth billing-classifier triggers fix(anthropic): alias session_search/memory OAuth billing-classifier triggers (fixes #65365, #82154) Aug 9, 2026
@JoaoMarcos44

Copy link
Copy Markdown
Contributor Author

Also fixes #82154

Independent bisection in #82154 landed on the same root cause with a narrower reproduction: the first sentence alone of SKILLS_GUIDANCE (the "5+ tool calls" sentence) trips Anthropic's OAuth billing classifier on its own, without needing the SESSION_SEARCH_GUIDANCE combination this PR originally bisected against.

The agent/prompt_builder.py change in this PR already supersedes the fix #82154 proposes:

Net effect: merging this PR closes both the 3-sentence combined trigger (#65365) and the single-sentence trigger (#82154) with one reworded prompt block, so no separate PR is needed for #82154 — opening a second one would just re-touch the same constant with a subset of this fix.

Flagging the two secondary observations from #82154 that are genuinely out of scope here and worth separate follow-ups:

  1. Exhausted-credential replay doesn't label cached vs. live errors during the ~60min cooldown (agent/credential_pool.py) — misleads iterative debugging.
  2. PROVIDER_REGISTRY["anthropic"].api_key_env_vars lists CLAUDE_CODE_OAUTH_TOKEN as if a setup-token value works as an API key; it 401s/429s instead. Needs a docstring note or removal from that tuple.

@Cloud-Ops-Dev — could you confirm on #82154 whether the reworded prose in this PR clears your live bisection harness too? Happy to adjust wording further if it doesn't.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint needs-decision Awaiting maintainer decision before any implementation P2 Medium — degraded but workaround exists provider/anthropic Anthropic native Messages API type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OAuth (Claude Pro/Max): exposing memory or session_search tool schema deterministically triggers HTTP 400 "You're out of extra usage"

2 participants