perf(prompt-caching): cache tool schemas without history loss

[rodboev]

Closes #20880

Summary

• Rebuild tool-schema cache placement as a request-local plan that allocates at most four combined tool and message breakpoints.
• Send a tool marker only to direct native Anthropic Messages requests, while preserving current message-cache behavior and plain tools for third-party and envelope routes.
• Replan both sections for normal retry, persistent MoA, and synchronous fallback destinations; configured fallbacks preserve their resolved provider, base URL, API mode, and model.

Validation

• 228 passed in 12.17s: cache plan, synchronous fallback, failover, MoA, policy, normal kwargs, and continuation compatibility tests.
• 28 passed in 1.16s: direct and third-party Anthropic-wire fallback payloads, persistent MoA, and cache policy boundaries.
• Invariant enumeration and proof-matrix validation pass.

Keeps the four-breakpoint budget from #27170 and incorporates @tbanetwork's current breakpoint-budget report.

[teknium1]

Thanks for targeting a real prompt-cost gap. Current main still caches only api_messages at agent/conversation_loop.py:894-899, while agent/chat_completion_helpers.py:777-792 forwards agent.tools to the request.

Problems

• The current PR commit (f5ddc911e10c) does not wire the new helper into that path: its only conversation_loop.py change is an import, and its chat_completion_helpers.py change only reads agent._tools_for_api. The complete PR diff contains no assignment to that attribute, so agent.tools remains the transmitted list without a marker.
• The new tests cover helper output but not request construction. They would not detect this missing invocation.
• The PR conflicts with main, and its prompt_caching.py hunk predates current _can_carry_marker filtering at agent/prompt_caching.py:109-117; salvage should retain that newer protection.

Suggested changes

• Apply tool marking to a request-local tool list in the actual request-preparation path, reserve a breakpoint only when that list is sent, and add native-Anthropic plus OpenAI-compatible request-kwargs coverage.

Automated hermes-sweeper review.

[kshitijk4poor]

Merged via #76032 — your commit was cherry-picked onto current main with authorship preserved (rebase merge), so it lands under your name:

• 9fc12bf perf(prompt-caching): preserve tool-loop cache boundaries (Tool-heavy agents pay ~70% input-token overhead — system_and_3 caching skips tools schema #20880)

Verified end-to-end before merge: budget invariant (<=4 markers) across adversarial message shapes, prefix stability across turns, byte-identical fallback behavior for non-native routes, and marker survival through the Anthropic wire conversion — plus 294 targeted tests green.

Thanks for the contribution, and for the thorough rework after the sweeper round (the request-local plan + destination replanning design turned out very clean).