Skip to content

fix(cron): use stable prompt cache key for recurring jobs, closes #51395 - #51942

Closed
manus-use wants to merge 1 commit into
NousResearch:mainfrom
manus-use:fix/cron-prompt-cache-warm
Closed

fix(cron): use stable prompt cache key for recurring jobs, closes #51395#51942
manus-use wants to merge 1 commit into
NousResearch:mainfrom
manus-use:fix/cron-prompt-cache-warm

Conversation

@manus-use

Copy link
Copy Markdown

Summary

Recurring cron jobs were prompt-cache-cold on every fire because session_id (used as the cache routing key) included a per-fire timestamp: cron_<job_id>_<YYYYMMDD_HHMMSS>. This meant a job firing every 5 minutes re-paid its full static prefix 288 times/day.

This PR introduces a prompt_cache_key parameter to the Responses API transport that defaults to session_id but can be overridden. The cron scheduler now sets agent.prompt_cache_key = 'cron_<job_id>' — a stable key that stays warm across fires.

Impact

Before After
Every cron fire = cache-cold (unique session_id per fire) First fire cold, subsequent fires hit warm cache
Job firing every 5min re-pays full prefix 288×/day Prefix paid once, reused indefinitely
Interactive sessions unaffected Interactive sessions unaffected (default = session_id)

Root Cause

# scheduler.py — session_id includes timestamp (changes every fire)
_cron_session_id = f"cron_{job_id}_{_hermes_now().strftime('%Y%m%d_%H%M%S')}"

codex.py::build_kwargs then used this as:

  • kwargs["prompt_cache_key"] (OpenAI standard)
  • Codex backend session_id/x-client-request-id headers
  • xAI extra_body["prompt_cache_key"]

All unique per fire → always cache-cold.

Fix

File Change
agent/transports/codex.py Added prompt_cache_key param to build_kwargs(). Uses it for all cache-routing fields. Falls back to session_id when not provided. session_id still used for xAI x-grok-conv-id (conversation tracking).
agent/chat_completion_helpers.py Passes agent.prompt_cache_key to build_kwargs() on the Codex/Responses path.
cron/scheduler.py Sets agent.prompt_cache_key = f"cron_{job_id}" after agent creation — stable across fires.
tests/agent/test_cron_prompt_cache_warm.py 10 tests covering override behavior, defaults, header routing, and integration.

Safety

As noted in the issue: prompt_cache_key is a routing hint, not a correctness boundary. A stale or wrong key only causes a cache miss, never a wrong result. This is a low-risk efficiency fix.

Backwards Compatibility

  • Interactive sessions: unchanged (no prompt_cache_key set → defaults to session_id)
  • Existing cron jobs: get warm cache immediately on next fire
  • No config changes needed

Test Results

tests/agent/test_cron_prompt_cache_warm.py: 10 passed
tests/agent/ (full suite): 2162 passed, 1 pre-existing failure (unrelated)
tests/agent/ -k "codex or transport": 505 passed

Closes #51395

…ch#51395)

Recurring cron jobs were prompt-cache-cold on every fire because the
session_id (used as the cache routing key) included a per-fire
timestamp: cron_<job_id>_<YYYYMMDD_HHMMSS>.

This commit introduces a prompt_cache_key parameter to the Responses
API transport that defaults to session_id but can be overridden. The
cron scheduler now sets agent.prompt_cache_key = 'cron_<job_id>' — a
stable key that stays warm across fires while session_id remains
unique for session tracking.

Impact: A job firing every 5 minutes was re-paying its full static
prefix (agent identity + tools + system prompt) 288 times/day.
Now it pays cold once and reuses the warm cache on subsequent fires.

Changes:
- agent/transports/codex.py: Add prompt_cache_key param to
  build_kwargs(); use it for prompt_cache_key body field, Codex
  backend cache-scope headers, and xAI extra_body cache routing.
  session_id still used for xAI x-grok-conv-id (conversation
  tracking). Defaults to session_id when not provided (no change
  for interactive sessions).
- agent/chat_completion_helpers.py: Pass agent.prompt_cache_key
  to build_kwargs() on the Codex/Responses path.
- cron/scheduler.py: Set agent.prompt_cache_key to the stable
  cron_<job_id> after agent creation.
- tests/agent/test_cron_prompt_cache_warm.py: 10 tests covering
  cache key override, defaults, header routing, and integration.
@alt-glitch alt-glitch added type/bug Something isn't working comp/cron Cron scheduler and job management comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P0 Critical — data loss, security, crash loop sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) duplicate This issue or pull request already exists labels Jun 24, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Duplicate of #51396 — same opt-in prompt_cache_key/cache_key mechanism (defaults to session_id, cron sets cron_<job_id> so the warm prefix persists across fires) for the same issue #51395. #51396 is the earlier canonical PR. Competing alternate mechanism #51585 (always-on content-addressed hash key) is related; a maintainer should pick between the opt-in-param cluster (#51396/this) and the content-hash approach (#51585).

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hermes Agent Review

Verdict: Approved — Clean, well-scoped change with appropriate tests.

Reviewed as part of batch review session 2026-06-24d.

Reviewed by Hermes Agent

@teknium1

Copy link
Copy Markdown
Contributor

Closing as a duplicate of the #51395 cluster — merged via #52295 (commit 7a65800). Same fix (stable prompt_cache_key for recurring cron jobs); we went with a content-addressed key. Thanks for the contribution!

@teknium1 teknium1 closed this Jun 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cron Cron scheduler and job management duplicate This issue or pull request already exists P0 Critical — data loss, security, crash loop sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Recurring cron jobs are prompt-cache-cold on every fire (session_id is the cache key)

4 participants