fix(cache): content-address prompt_cache_key so recurring cron jobs reuse the warm prefix by teknium1 · Pull Request #52295 · NousResearch/hermes-agent

teknium1 · 2026-06-25T03:41:59Z

Summary

Recurring cron jobs now share one stable, content-addressed prompt_cache_key across fires instead of going cache-cold every tick.

Root cause: cron builds session_id as cron_<job_id>_<timestamp>, and the Codex/Responses transport used session_id directly as prompt_cache_key. The timestamp changed the key on every fire, so the static prefix (agent identity + tool schemas) was re-tokenised cold each run. Closes #51395.

Changes

agent/transports/codex.py: new _content_cache_key(instructions, tools) returning pck_<sha256[:24]> of the static prefix (tools sorted by name; \x00 separator avoids instruction/tool boundary collisions). prompt_cache_key (standard body field + xAI extra_body) now uses it, falling back to session_id only when there is no static content.
session_id is unchanged for transcript isolation, logs, and the session-scope routing headers (session_id / x-client-request-id / x-grok-conv-id) — those keep their per-fire identity. Only the cache key is content-addressed.
Tests updated to assert the new contract.

Scope note

This is the minimal version. The cache key is a routing hint, never a correctness boundary — a stale/shared key can only cause a miss, never a wrong result. The win is real but bounded by provider cache TTL (~5-60 min), so it primarily helps sub-TTL-interval jobs; longer cadences are cold by the next fire regardless. Kept deliberately small for that reason.

Validation

	Before	After
Two fires, same job (diff timestamp)	different key, cold	same `pck_` key, warm
Cache key value	raw `session_id`	content hash
Different job prompt	n/a	different key (no collision)
Codex transcript headers	`session_id`	`session_id` (unchanged)

E2E-verified with real transport imports; tests/agent/transports/test_codex_transport.py 60/60 green.

Credit

Closes the cluster on #51395. Fix scoped by @spiky02plateau (#51396, issue author); content-hash key naming from @JoaoMarcos44 (#51585). Both credited as co-authors.

Infographic

…euse the warm prefix Recurring cron jobs were prompt-cache-cold on every fire. session_id is built as cron_<job_id>_<timestamp>, and the Codex/Responses transport used session_id directly as prompt_cache_key — so the timestamp changed the cache key on every run and the static prefix (agent identity + tool schemas) was re-paid each tick. Derive prompt_cache_key from a SHA-256 of the static prefix (instructions + sorted tool schemas) instead. Repeated fires of the same job share one content-addressed key (pck_<hash>) and reuse the warm prefix within the provider's cache TTL. The key changes exactly when the prefix changes — edit the job's prompt or toolset and it re-keys; leave it alone and it stays stable. session_id is left untouched for transcript isolation, log correlation, and the Codex/xAI session-scope routing headers (session_id, x-client-request-id, x-grok-conv-id) — those are the per-fire identity, not the cache key. Only the prompt_cache_key body field (standard OpenAI/Codex path and the xAI extra_body field) is content-addressed. Closes #51395. Co-authored-by: spiky02plateau <spiky02plateau@users.noreply.github.com> Co-authored-by: JoaoMarcos44 <JoaoMarcos44@users.noreply.github.com>

github-actions · 2026-06-25T03:42:55Z

🔎 Lint report: `fix/cron-cache-key-content-hash` vs `origin/main`

ruff

Total: 0 on HEAD, 0 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 0 pre-existing issues carried over.

ty (type checker)

Total: 11261 on HEAD, 11259 on base (🆕 +2)

🆕 New issues (2):

Rule	Count
`unresolved-attribute`	2

First entries

tests/run_agent/test_credits_notices_toggle.py:76: [unresolved-attribute] unresolved-attribute: Unresolved attribute `_credits_session_start_micros` on type `AIAgent`
run_agent.py:2984: [unresolved-attribute] unresolved-attribute: Object of type `Self@get_credits_spent_micros` has no attribute `_credits_session_start_micros`

✅ Fixed issues (1):

Rule	Count
`invalid-assignment`	1

First entries

tests/run_agent/test_credits_notices_toggle.py:76: [invalid-assignment] invalid-assignment: Object of type `None` is not assignable to attribute `_credits_session_start_micros` of type `int`

Unchanged: 5940 pre-existing issues carried over.

Diagnostics are surfaced as warnings — this check never fails the build.

tonydwb

Code Review Summary

Verdict: Approved

Clean fix for content-addressed prompt cache key in cron jobs. The solution correctly hashes the static prefix (instructions + tools) to produce a stable cache key across fires, while preserving session_id for transcript isolation. Good test coverage.

Changes:

agent/transports/codex.py: New _content_cache_key function, updated build_kwargs to use content-addressed key
tests/agent/transports/test_codex_transport.py: Updated tests for new cache key behavior

Well-documented with clear before/after table. LGTM.

Reviewed by Hermes Agent

…euse the warm prefix (NousResearch#52295) Recurring cron jobs were prompt-cache-cold on every fire. session_id is built as cron_<job_id>_<timestamp>, and the Codex/Responses transport used session_id directly as prompt_cache_key — so the timestamp changed the cache key on every run and the static prefix (agent identity + tool schemas) was re-paid each tick. Derive prompt_cache_key from a SHA-256 of the static prefix (instructions + sorted tool schemas) instead. Repeated fires of the same job share one content-addressed key (pck_<hash>) and reuse the warm prefix within the provider's cache TTL. The key changes exactly when the prefix changes — edit the job's prompt or toolset and it re-keys; leave it alone and it stays stable. session_id is left untouched for transcript isolation, log correlation, and the Codex/xAI session-scope routing headers (session_id, x-client-request-id, x-grok-conv-id) — those are the per-fire identity, not the cache key. Only the prompt_cache_key body field (standard OpenAI/Codex path and the xAI extra_body field) is content-addressed. Closes NousResearch#51395. Co-authored-by: spiky02plateau <spiky02plateau@users.noreply.github.com> Co-authored-by: JoaoMarcos44 <JoaoMarcos44@users.noreply.github.com>

…the warm prefix Port of NousResearch/hermes-agent 7a65800 (NousResearch#52295). Every cron fire (cron_{job}_{ts}) and parallel quorum panelist got a unique session_id prompt_cache_key, defeating provider prefix-cache reuse of the large static soul/protocol/tool prefix. Now the key is _content_cache_key(instructions, tools) = pck_<sha256[:24]> so runs sharing the prefix warm each other (xAI extra_body path too); codex-backend headers keep session_id for per-fire identity. 52 codex-transport tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…euse the warm prefix (NousResearch#52295) Recurring cron jobs were prompt-cache-cold on every fire. session_id is built as cron_<job_id>_<timestamp>, and the Codex/Responses transport used session_id directly as prompt_cache_key — so the timestamp changed the cache key on every run and the static prefix (agent identity + tool schemas) was re-paid each tick. Derive prompt_cache_key from a SHA-256 of the static prefix (instructions + sorted tool schemas) instead. Repeated fires of the same job share one content-addressed key (pck_<hash>) and reuse the warm prefix within the provider's cache TTL. The key changes exactly when the prefix changes — edit the job's prompt or toolset and it re-keys; leave it alone and it stays stable. session_id is left untouched for transcript isolation, log correlation, and the Codex/xAI session-scope routing headers (session_id, x-client-request-id, x-grok-conv-id) — those are the per-fire identity, not the cache key. Only the prompt_cache_key body field (standard OpenAI/Codex path and the xAI extra_body field) is content-addressed. Closes NousResearch#51395. Co-authored-by: spiky02plateau <spiky02plateau@users.noreply.github.com> Co-authored-by: JoaoMarcos44 <JoaoMarcos44@users.noreply.github.com>

…ing key Nous Portal and OpenRouter provider profiles pin every turn of a session to the same upstream endpoint (body["session_id"]) so Anthropic/Vertex/Bedrock cache_control breakpoints stay warm. That key came straight from get_conversation_context() or session_id with no normalization, so cron re-fires (cron_<job_id>_<timestamp>, no parent_session_id to walk) got a fresh key every run and never pinned to the same endpoint -- the same NousResearch#51395/NousResearch#52295 class of bug the original prompt_cache_key fix addressed, just on a route NousResearch#78959 didn't touch. Reuses _cache_scope_from_session_id() (no new logic) and leaves the Portal's conversation= analytics tag and the xAI x-grok-conv-id header untouched, since those need per-fire identity, not cache affinity.

…euse the warm prefix (NousResearch#52295) Recurring cron jobs were prompt-cache-cold on every fire. session_id is built as cron_<job_id>_<timestamp>, and the Codex/Responses transport used session_id directly as prompt_cache_key — so the timestamp changed the cache key on every run and the static prefix (agent identity + tool schemas) was re-paid each tick. Derive prompt_cache_key from a SHA-256 of the static prefix (instructions + sorted tool schemas) instead. Repeated fires of the same job share one content-addressed key (pck_<hash>) and reuse the warm prefix within the provider's cache TTL. The key changes exactly when the prefix changes — edit the job's prompt or toolset and it re-keys; leave it alone and it stays stable. session_id is left untouched for transcript isolation, log correlation, and the Codex/xAI session-scope routing headers (session_id, x-client-request-id, x-grok-conv-id) — those are the per-fire identity, not the cache key. Only the prompt_cache_key body field (standard OpenAI/Codex path and the xAI extra_body field) is content-addressed. Closes NousResearch#51395. Co-authored-by: spiky02plateau <spiky02plateau@users.noreply.github.com> Co-authored-by: JoaoMarcos44 <JoaoMarcos44@users.noreply.github.com>

tonydwb approved these changes Jun 25, 2026

View reviewed changes

teknium1 merged commit 7a65800 into main Jun 25, 2026
27 checks passed

teknium1 deleted the fix/cron-cache-key-content-hash branch June 25, 2026 04:46

This was referenced Jun 27, 2026

MoA auxiliary Codex Responses calls lack stable prompt_cache_key/cache affinity #53735

Closed

fix(moa): add stable prompt cache key for auxiliary Codex Responses calls #53737

Closed

RyderFreeman4Logos mentioned this pull request Aug 4, 2026

[Cache]: content-only prompt_cache_key can concentrate unrelated sessions into one routing scope #78941

Closed

StanleyStetson mentioned this pull request Aug 4, 2026

fix(agent): namespace prompt_cache_key with logical session scope #78956

Closed

JoaoMarcos44 mentioned this pull request Aug 4, 2026

fix(cache): scope prompt_cache_key by session to stop cross-session bucket sharing #78959

Closed

5 tasks

JoaoMarcos44 mentioned this pull request Aug 5, 2026

x-grok-conv-id unstable across cron re-fires of the same job (native xAI Responses and OpenRouter) #79015

Closed

Navigation Menu

Uh oh!

fix(cache): content-address prompt_cache_key so recurring cron jobs reuse the warm prefix - #52295

fix(cache): content-address prompt_cache_key so recurring cron jobs reuse the warm prefix#52295
teknium1 merged 1 commit into
mainfrom
fix/cron-cache-key-content-hash

teknium1 commented Jun 25, 2026

Uh oh!

github-actions Bot commented Jun 25, 2026

Uh oh!

tonydwb left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

teknium1 commented Jun 25, 2026

Summary

Changes

Scope note

Validation

Credit

Infographic

Uh oh!

github-actions Bot commented Jun 25, 2026

🔎 Lint report: fix/cron-cache-key-content-hash vs origin/main

ruff

ty (type checker)

Uh oh!

tonydwb left a comment

Choose a reason for hiding this comment

Code Review Summary

Well-documented with clear before/after table. LGTM.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

🔎 Lint report: `fix/cron-cache-key-content-hash` vs `origin/main`