fix(caching): prompt_caching.enabled toggle for strict Anthropic proxies (salvage #35862, closes #13477) by teknium1 · Pull Request #56105 · NousResearch/hermes-agent

teknium1 · 2026-07-01T06:34:43Z

Summary

Adds a prompt_caching.enabled config toggle so a user on a strict Anthropic-compatible proxy (LiteLLM-style, e.g. llm.echo.tech) that injects its own cache_control markers can turn off client-side markers entirely — fixing the HTTP 400 "A maximum of 4 blocks with cache_control may be provided. Found 5" reported in #13477 without regressing caching for every well-behaved third-party gateway.

Salvages @janrenz's PR #35862 (the config toggle), then closes the gap it left: the disable must survive /model switches and fallback re-derivation.

Why the toggle, not a third-party strip

The original report (#13477) proposed stripping ALL markers whenever base_url isn't anthropic.com. That matches every non-Anthropic endpoint, so it would kill caching for MiniMax, Zhipu GLM, Bedrock, Foundry, Kimi, DeepSeek — all of which anthropic_prompt_cache_policy deliberately caches on and none of which double-inject. Wrong lever. The real problem is one misbehaving proxy adding a 5th marker on top of our 4; the correct fix is a per-setup opt-out.

Changes

hermes_cli/config.py: add prompt_caching.enabled: true default + escape-hatch comment.
agent/agent_runtime_helpers.py: gate anthropic_prompt_cache_policy on enabled=false → returns (False, False) before any branch, so init, /model switch, and fallback all honor it.
agent/agent_init.py: keep @janrenz's isinstance hardening on the cache_ttl read; drop the now-redundant init-only override (the policy gate covers it).
tests/: policy-level kill-switch tests (native / OpenRouter / third-party / model-switch survival / enabled-true-keeps-caching) + @janrenz's init test.
website/docs/: document the enabled toggle.
scripts/release.py: AUTHOR_MAP entry for @janrenz.

Root cause

Client never exceeds 4 markers on native Anthropic (traced E2E: apply_anthropic_cache_control marks system + last-3, tool-merge collapses them 1:1). The proxy adds a 5th server-side. The only client-side lever that helps is not sending markers to that proxy — a config toggle.

Validation

path	enabled=false	enabled=true (default)
native Anthropic	(False, False)	(True, True)
OpenRouter Claude	(False, False)	(True, False)
third-party proxy	(False, False)	(True, True)
after `/model` switch	(False, False)	policy-derived
after fallback swap	(False, False)	policy-derived

E2E confirmed the disable survives all three re-derivation paths and re-enabling restores caching. test_anthropic_prompt_cache_policy.py 29/29 green; test_run_agent.py caching subset 9/9 green.

Closes #13477 (via the toggle, not the proposed strip). Salvages #35862 with authorship preserved.

Infographic

@janrenz

…back @janrenz's PR #35862 added prompt_caching.enabled=false at init only. But _anthropic_prompt_cache_policy re-derives _use_prompt_caching on every /model switch (agent_runtime_helpers) and fallback-model swap (chat_completion_helpers), which re-enabled markers and re-broke the strict proxy the toggle was meant to fix. Move the kill switch into anthropic_prompt_cache_policy so it returns (False, False) on every path. Drop the now-redundant init-time override (kept @janrenz's isinstance hardening on the cache_ttl read). Add policy-level tests + docs for the toggle. Follow-up to salvaged PR #35862.

alt-glitch · 2026-07-01T06:48:53Z

This was generated by AI during triage.

Salvage of @janrenz's #35862 (the prompt_caching.enabled toggle) closing #13477, extended so the disable survives /model switches and fallback re-derivation. Related, not a duplicate.

…tion (#56126) * Revert "fix(caching): honor prompt_caching.enabled across model switch + fallback" This reverts commit 36f9f50. * Revert "fix: allow disabling prompt caching" This reverts commit c1c1a12.

…or re-evaluation (NousResearch#56126) * Revert "fix(caching): honor prompt_caching.enabled across model switch + fallback" This reverts commit 1f86e39. * Revert "fix: allow disabling prompt caching" This reverts commit 7e81b23.

…or re-evaluation (NousResearch#56126) * Revert "fix(caching): honor prompt_caching.enabled across model switch + fallback" This reverts commit 36f9f50. * Revert "fix: allow disabling prompt caching" This reverts commit c1c1a12.

…or re-evaluation (NousResearch#56126) * Revert "fix(caching): honor prompt_caching.enabled across model switch + fallback" This reverts commit 969477b. * Revert "fix: allow disabling prompt caching" This reverts commit 58d8548.

…or re-evaluation (NousResearch#56126) * Revert "fix(caching): honor prompt_caching.enabled across model switch + fallback" This reverts commit c5b2feb. * Revert "fix: allow disabling prompt caching" This reverts commit 64a756d.

…or re-evaluation (NousResearch#56126) * Revert "fix(caching): honor prompt_caching.enabled across model switch + fallback" This reverts commit 36f9f50. * Revert "fix: allow disabling prompt caching" This reverts commit c1c1a12.

@BB-light

Setting prompt_caching.cache_ttl to a falsy value (false, null, off, disabled, no, none) now fully disables prompt caching instead of being silently ignored. The disable propagates through anthropic_prompt_cache_policy() (early return when _cache_disabled flag is set) and restore_primary_runtime() (override after snapshot restore), so it survives /model switches and fallback re-derivation — the gap that caused NousResearch#56105 to be reverted in NousResearch#56126. Salvage of NousResearch#33555 by @BB-light, with model-switch/fallback survival gap fixed on top. Co-authored-by: BB-light <BB-light@users.noreply.github.com>

@BB-light

Setting prompt_caching.cache_ttl to a falsy value (false, null, off, disabled, no, none) now fully disables prompt caching instead of being silently ignored. The disable propagates through anthropic_prompt_cache_policy() (early return when _cache_disabled flag is set) and restore_primary_runtime() (override after snapshot restore), so it survives /model switches and fallback re-derivation — the gap that caused #56105 to be reverted in #56126. Salvage of #33555 by @BB-light, with model-switch/fallback survival gap fixed on top. Co-authored-by: BB-light <BB-light@users.noreply.github.com>

…or re-evaluation (NousResearch#56126) * Revert "fix(caching): honor prompt_caching.enabled across model switch + fallback" This reverts commit 36f9f50. * Revert "fix: allow disabling prompt caching" This reverts commit c1c1a12.

@BB-light

Setting prompt_caching.cache_ttl to a falsy value (false, null, off, disabled, no, none) now fully disables prompt caching instead of being silently ignored. The disable propagates through anthropic_prompt_cache_policy() (early return when _cache_disabled flag is set) and restore_primary_runtime() (override after snapshot restore), so it survives /model switches and fallback re-derivation — the gap that caused NousResearch#56105 to be reverted in NousResearch#56126. Salvage of NousResearch#33555 by @BB-light, with model-switch/fallback survival gap fixed on top. Co-authored-by: BB-light <BB-light@users.noreply.github.com>

@BB-light

Setting prompt_caching.cache_ttl to a falsy value (false, null, off, disabled, no, none) now fully disables prompt caching instead of being silently ignored. The disable propagates through anthropic_prompt_cache_policy() (early return when _cache_disabled flag is set) and restore_primary_runtime() (override after snapshot restore), so it survives /model switches and fallback re-derivation — the gap that caused NousResearch#56105 to be reverted in NousResearch#56126. Salvage of NousResearch#33555 by @BB-light, with model-switch/fallback survival gap fixed on top. Co-authored-by: BB-light <BB-light@users.noreply.github.com>

…or re-evaluation (NousResearch#56126) * Revert "fix(caching): honor prompt_caching.enabled across model switch + fallback" This reverts commit 36f9f50. * Revert "fix: allow disabling prompt caching" This reverts commit c1c1a12.

@BB-light

Setting prompt_caching.cache_ttl to a falsy value (false, null, off, disabled, no, none) now fully disables prompt caching instead of being silently ignored. The disable propagates through anthropic_prompt_cache_policy() (early return when _cache_disabled flag is set) and restore_primary_runtime() (override after snapshot restore), so it survives /model switches and fallback re-derivation — the gap that caused NousResearch#56105 to be reverted in NousResearch#56126. Salvage of NousResearch#33555 by @BB-light, with model-switch/fallback survival gap fixed on top. Co-authored-by: BB-light <BB-light@users.noreply.github.com>

Jan Renz and others added 2 commits June 30, 2026 23:26

fix: allow disabling prompt caching

4d0b114

teknium1 merged commit 36f9f50 into main Jul 1, 2026
31 checks passed

teknium1 deleted the hermes/hermes-eba48a36 branch July 1, 2026 07:10

This was referenced Jul 1, 2026

fix: allow disabling prompt caching #35862

Closed

fix(anthropic): prevent cache_control overload with LiteLLM-style proxies #13477

Closed

revert: back out prompt_caching.enabled toggle (#56105) for re-evaluation #56126

Merged

teknium1 mentioned this pull request Jul 1, 2026

[FOR REVIEW] prompt_caching.enabled toggle for strict Anthropic proxies (re-open of #56105) #56134

Closed

hashbender mentioned this pull request Jul 1, 2026

fix(caching): prompt_caching.enabled toggle for strict Anthropic proxies (salvage #35862, closes #13477) hashbender/hermes-agent#184

Merged

GottZ mentioned this pull request Jul 29, 2026

Prompt caching: add DeepSeek models to cache_control whitelist for OpenCode Go #24617

Closed

kshitijk4poor mentioned this pull request Aug 1, 2026

fix(caching): honor prompt_caching.cache_ttl disable in config (salvage #33555) #76015

Merged

kshitijk4poor mentioned this pull request Aug 1, 2026

fix(caching): honor prompt_caching.cache_ttl disable in config #33555

Closed

5 tasks

Navigation Menu

fix(caching): prompt_caching.enabled toggle for strict Anthropic proxies (salvage #35862, closes #13477) - #56105

fix(caching): prompt_caching.enabled toggle for strict Anthropic proxies (salvage #35862, closes #13477)#56105
teknium1 merged 2 commits into
mainfrom
hermes/hermes-eba48a36

teknium1 commented Jul 1, 2026

Uh oh!

alt-glitch commented Jul 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

teknium1 commented Jul 1, 2026

Summary

Why the toggle, not a third-party strip

Changes

Root cause

Validation

Infographic

Uh oh!

alt-glitch commented Jul 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants