fix: allow disabling prompt caching - #35862
Closed
janrenz wants to merge 1 commit into
Closed
Conversation
alt-glitch
added
type/feature
Collaborator
|
Related: #33555 addresses the same problem (disabling prompt caching) via a different approach — falsy |
tonydwb
approved these changes
May 31, 2026
tonydwb
left a comment
tonydwb
left a comment
There was a problem hiding this comment.
Code Review Summary
Verdict: Approved ⚡
✅ What's Good
- Necessary escape hatch: Some strict Anthropic-compatible endpoints reject
cache_controlmarkers. Addingprompt_caching.enabled: falsegives users a config-gated way to disable it without breaking existing behavior (default staystrue). - Minimal change: 32 additions, 2 deletions. Well-scoped.
- Backward compatible: Default is
true, so existing configs work unchanged. - Safe guard: The
isinstance(_pc_cfg, dict)check prevents crashes on unexpected config shapes. - Good test: Verifies both
_use_prompt_cachingand_use_native_cache_layoutare disabled, which is the complete effect.
Summary
A simple, well-scoped config escape hatch. No issues found.
Reviewed by Hermes Agent
teknium1
added a commit
that referenced
this pull request
Jul 1, 2026
…back @janrenz's PR #35862 added prompt_caching.enabled=false at init only. But _anthropic_prompt_cache_policy re-derives _use_prompt_caching on every /model switch (agent_runtime_helpers) and fallback-model swap (chat_completion_helpers), which re-enabled markers and re-broke the strict proxy the toggle was meant to fix. Move the kill switch into anthropic_prompt_cache_policy so it returns (False, False) on every path. Drop the now-redundant init-time override (kept @janrenz's isinstance hardening on the cache_ttl read). Add policy-level tests + docs for the toggle. Follow-up to salvaged PR #35862.
Contributor
|
Salvaged and merged via #56105 (rebase-merge, your commit's authorship preserved in git log on main). Your prompt_caching.enabled toggle is exactly the right lever for strict Anthropic-compatible proxies. Follow-up on top moved the gate into anthropic_prompt_cache_policy so the disable survives /model switch and fallback re-derivation (init-only wasn't enough), plus docs and policy tests. Thanks! |
hashbender
added a commit
to hashbender/hermes-agent
that referenced
this pull request
Jul 1, 2026
…ies (salvage NousResearch#35862, closes NousResearch#13477) (#184) Co-authored-by: qbit-mirror-bot <qbit-mirror-bot@users.noreply.github.com>
waefrebeorn
pushed a commit
to waefrebeorn/slermes
that referenced
this pull request
Jul 2, 2026
…back @janrenz's PR NousResearch#35862 added prompt_caching.enabled=false at init only. But _anthropic_prompt_cache_policy re-derives _use_prompt_caching on every /model switch (agent_runtime_helpers) and fallback-model swap (chat_completion_helpers), which re-enabled markers and re-broke the strict proxy the toggle was meant to fix. Move the kill switch into anthropic_prompt_cache_policy so it returns (False, False) on every path. Drop the now-redundant init-time override (kept @janrenz's isinstance hardening on the cache_ttl read). Add policy-level tests + docs for the toggle. Follow-up to salvaged PR NousResearch#35862.
Jasper6439
pushed a commit
to Jasper6439/hermes-agent
that referenced
this pull request
Jul 5, 2026
…back @janrenz's PR NousResearch#35862 added prompt_caching.enabled=false at init only. But _anthropic_prompt_cache_policy re-derives _use_prompt_caching on every /model switch (agent_runtime_helpers) and fallback-model swap (chat_completion_helpers), which re-enabled markers and re-broke the strict proxy the toggle was meant to fix. Move the kill switch into anthropic_prompt_cache_policy so it returns (False, False) on every path. Drop the now-redundant init-time override (kept @janrenz's isinstance hardening on the cache_ttl read). Add policy-level tests + docs for the toggle. Follow-up to salvaged PR NousResearch#35862.
teknium1
added a commit
that referenced
this pull request
Jul 6, 2026
…back @janrenz's PR #35862 added prompt_caching.enabled=false at init only. But _anthropic_prompt_cache_policy re-derives _use_prompt_caching on every /model switch (agent_runtime_helpers) and fallback-model swap (chat_completion_helpers), which re-enabled markers and re-broke the strict proxy the toggle was meant to fix. Move the kill switch into anthropic_prompt_cache_policy so it returns (False, False) on every path. Drop the now-redundant init-time override (kept @janrenz's isinstance hardening on the cache_ttl read). Add policy-level tests + docs for the toggle. Follow-up to salvaged PR #35862. (cherry picked from commit 36f9f50)
habarmc1223-sudo
pushed a commit
to habarmc1223-sudo/hermes-agent-fluxmem
that referenced
this pull request
Jul 8, 2026
…back @janrenz's PR NousResearch#35862 added prompt_caching.enabled=false at init only. But _anthropic_prompt_cache_policy re-derives _use_prompt_caching on every /model switch (agent_runtime_helpers) and fallback-model swap (chat_completion_helpers), which re-enabled markers and re-broke the strict proxy the toggle was meant to fix. Move the kill switch into anthropic_prompt_cache_policy so it returns (False, False) on every path. Drop the now-redundant init-time override (kept @janrenz's isinstance hardening on the cache_ttl read). Add policy-level tests + docs for the toggle. Follow-up to salvaged PR NousResearch#35862.
santhreal
pushed a commit
to santhreal/hermes-agent
that referenced
this pull request
Jul 13, 2026
…back @janrenz's PR NousResearch#35862 added prompt_caching.enabled=false at init only. But _anthropic_prompt_cache_policy re-derives _use_prompt_caching on every /model switch (agent_runtime_helpers) and fallback-model swap (chat_completion_helpers), which re-enabled markers and re-broke the strict proxy the toggle was meant to fix. Move the kill switch into anthropic_prompt_cache_policy so it returns (False, False) on every path. Drop the now-redundant init-time override (kept @janrenz's isinstance hardening on the cache_ttl read). Add policy-level tests + docs for the toggle. Follow-up to salvaged PR NousResearch#35862.
Gravezzz
pushed a commit
to Gravezzz/hermes-agent
that referenced
this pull request
Jul 21, 2026
…back @janrenz's PR NousResearch#35862 added prompt_caching.enabled=false at init only. But _anthropic_prompt_cache_policy re-derives _use_prompt_caching on every /model switch (agent_runtime_helpers) and fallback-model swap (chat_completion_helpers), which re-enabled markers and re-broke the strict proxy the toggle was meant to fix. Move the kill switch into anthropic_prompt_cache_policy so it returns (False, False) on every path. Drop the now-redundant init-time override (kept @janrenz's isinstance hardening on the cache_ttl read). Add policy-level tests + docs for the toggle. Follow-up to salvaged PR NousResearch#35862.
leewenjie
pushed a commit
to leewenjie/hermes-agent
that referenced
this pull request
Aug 7, 2026
…back @janrenz's PR NousResearch#35862 added prompt_caching.enabled=false at init only. But _anthropic_prompt_cache_policy re-derives _use_prompt_caching on every /model switch (agent_runtime_helpers) and fallback-model swap (chat_completion_helpers), which re-enabled markers and re-broke the strict proxy the toggle was meant to fix. Move the kill switch into anthropic_prompt_cache_policy so it returns (False, False) on every path. Drop the now-redundant init-time override (kept @janrenz's isinstance hardening on the cache_ttl read). Add policy-level tests + docs for the toggle. Follow-up to salvaged PR NousResearch#35862.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
prompt_caching.enabledconfig support, defaulting totrueprompt_caching.enabled: falseto disable cache marker injection for strict Anthropic-compatible providersWhy
Some strict Anthropic-compatible endpoints can reject Hermes-injected
cache_controlmarkers with errors likeUnknown parameter: messages[0].content[0].cache_control. Hermes already supportsprompt_caching.cache_ttl; this adds a narrow config-gated escape hatch without changing the default behavior for users who benefit from prompt caching.Test Plan
venv/bin/python -m py_compile agent/agent_init.py hermes_cli/config.py tests/run_agent/test_run_agent.pyvenv/bin/python -m pytest tests/run_agent/test_run_agent.py tests/run_agent/test_anthropic_prompt_cache_policy.py tests/agent/test_prompt_caching.py -q -o 'addopts='Result: 396 passed, 1 warning.