fix: enforce Anthropic cache_control budget

[1justahead]

Summary

• stop forwarding tool-level cache_control markers into native Anthropic tool definitions
• add a final request-wide cache-control budget sanitizer for Anthropic kwargs
• add regression tests for tool marker stripping and the request-wide 4-block limit

Why

Anthropic enforces a hard request-wide limit of 4 cache_control blocks across system, messages, and tools. Hermes' prompt caching strategy already uses the full budget for system + recent messages. If a tool/schema marker is forwarded as well, native Anthropic rejects the request with:

HTTP 400: A maximum of 4 blocks with cache_control may be provided. Found 5.

The sanitizer strips extras before the SDK call, preferring tools first because tool-schema caching is not currently part of Hermes' budgeted placement strategy.

Test plan

• python3 -m pytest tests/agent/test_prompt_caching.py -q

Result: 16 passed

[teknium1]

Thanks for the focused fix. I verified the premise against current main and did not find correctness or design blockers in the PR diff.

Current main still has the over-budget path: agent/prompt_caching.py:56 documents the four-marker strategy, agent/anthropic_adapter.py:1529 still forwards tool-level cache_control into Anthropic tool definitions, and agent/anthropic_adapter.py:2473 returns the combined kwargs without a request-wide clamp. Those kwargs reach the native SDK stream call at agent/chat_completion_helpers.py:2153.

The PR’s approach fits the existing narrow adapter boundary: it removes tool-schema marker forwarding and adds a final Anthropic kwargs sanitizer in build_anthropic_kwargs rather than adding new user-facing config or model-tool surface. The follow-up commit f1dbbf3cc2f294bcc8af431684a45b50ed7045ad also addresses the earlier review concern by asserting stripping priority, not only the final count.

Automated hermes-sweeper review.

[teknium1]

Thanks for the focused cache-budget fix. The current-main premise is real: agent/prompt_caching.py:89-97 creates up to four markers, while agent/anthropic_adapter.py:1690-1695 can forward a tool-schema marker.

Problems

• agent/anthropic_adapter.py:1314 removes tool-schema cache_control unconditionally. Current main explicitly supports that marker to cache the schema cross-session at agent/anthropic_adapter.py:1690-1695; this changes supported caching behavior even when the request is within budget.
• _count_cache_control() at agent/anthropic_adapter.py:1891 recursively treats every dict-valued cache_control key as a request marker. A valid input_schema.properties.cache_control object would be counted and can be deleted by _strip_first_cache_control() at line 1876.

Suggested changes

• Retain legitimate schema caching and constrain budget accounting/removal to valid Anthropic marker positions.
• Add a regression case for an input-schema property named cache_control, alongside the request-wide cap test.

Automated hermes-sweeper review.

[teknium1]

This unconditionally removes a supported schema-cache marker. Current main deliberately forwards it at agent/anthropic_adapter.py:1690-1695 to cache the tool schema cross-session. Please enforce the four-marker budget without disabling schema caching for requests that do not exceed it.

[teknium1]

This counts any dict-valued key named cache_control, including a valid JSON Schema field such as input_schema.properties.cache_control. If the budget is exceeded, _strip_first_cache_control() removes that schema property. Restrict detection to valid Anthropic cache-marker locations and add a schema-property regression test.

[kshitijk4poor]

Closing as superseded by #76032 (merged), which lands tool-schema caching via a request-local breakpoint planner (build_prompt_cache_plan, salvaged from #37611 / closes #20880).

This PR and that planner take opposite positions on the same budget problem: this PR strips tool-level cache_control and adds a request-wide sanitizer to stay under Anthropic's 4-block limit; the merged planner instead allocates the budget deliberately (static system prefix + tools[-1] + last-2 transaction endpoints, provably <=4 by construction, enforced by tests and probes) so the tool marker is both legal and load-bearing. Stripping it now would undo the #20880 fix — tool-heavy agents would go back to re-sending ~12K tokens of schema uncached on every call.

The sweeper's earlier notes on this PR also stand: the unconditional strip removed supported cross-session schema caching, and the recursive _count_cache_control could miscount/delete an input_schema.properties.cache_control field.

The underlying concern (never exceed the 4-block budget) is real and now guarded where the markers are planned rather than sanitized after the fact. Thanks for the report and the fix — the budget framing helped shape the review of the merged design.