fix(cron): use stable prompt cache key for recurring jobs, closes #51395 - #51942
Closed
manus-use wants to merge 1 commit into
Closed
fix(cron): use stable prompt cache key for recurring jobs, closes #51395#51942manus-use wants to merge 1 commit into
manus-use wants to merge 1 commit into
Conversation
…ch#51395) Recurring cron jobs were prompt-cache-cold on every fire because the session_id (used as the cache routing key) included a per-fire timestamp: cron_<job_id>_<YYYYMMDD_HHMMSS>. This commit introduces a prompt_cache_key parameter to the Responses API transport that defaults to session_id but can be overridden. The cron scheduler now sets agent.prompt_cache_key = 'cron_<job_id>' — a stable key that stays warm across fires while session_id remains unique for session tracking. Impact: A job firing every 5 minutes was re-paying its full static prefix (agent identity + tools + system prompt) 288 times/day. Now it pays cold once and reuses the warm cache on subsequent fires. Changes: - agent/transports/codex.py: Add prompt_cache_key param to build_kwargs(); use it for prompt_cache_key body field, Codex backend cache-scope headers, and xAI extra_body cache routing. session_id still used for xAI x-grok-conv-id (conversation tracking). Defaults to session_id when not provided (no change for interactive sessions). - agent/chat_completion_helpers.py: Pass agent.prompt_cache_key to build_kwargs() on the Codex/Responses path. - cron/scheduler.py: Set agent.prompt_cache_key to the stable cron_<job_id> after agent creation. - tests/agent/test_cron_prompt_cache_warm.py: 10 tests covering cache key override, defaults, header routing, and integration.
Collaborator
Duplicate of #51396 — same opt-in |
tonydwb
approved these changes
Jun 24, 2026
tonydwb
left a comment
There was a problem hiding this comment.
Hermes Agent Review
Verdict: Approved — Clean, well-scoped change with appropriate tests.
Reviewed as part of batch review session 2026-06-24d.
Reviewed by Hermes Agent
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Recurring cron jobs were prompt-cache-cold on every fire because
session_id(used as the cache routing key) included a per-fire timestamp:cron_<job_id>_<YYYYMMDD_HHMMSS>. This meant a job firing every 5 minutes re-paid its full static prefix 288 times/day.This PR introduces a
prompt_cache_keyparameter to the Responses API transport that defaults tosession_idbut can be overridden. The cron scheduler now setsagent.prompt_cache_key = 'cron_<job_id>'— a stable key that stays warm across fires.Impact
Root Cause
codex.py::build_kwargsthen used this as:kwargs["prompt_cache_key"](OpenAI standard)session_id/x-client-request-idheadersextra_body["prompt_cache_key"]All unique per fire → always cache-cold.
Fix
agent/transports/codex.pyprompt_cache_keyparam tobuild_kwargs(). Uses it for all cache-routing fields. Falls back tosession_idwhen not provided.session_idstill used for xAIx-grok-conv-id(conversation tracking).agent/chat_completion_helpers.pyagent.prompt_cache_keytobuild_kwargs()on the Codex/Responses path.cron/scheduler.pyagent.prompt_cache_key = f"cron_{job_id}"after agent creation — stable across fires.tests/agent/test_cron_prompt_cache_warm.pySafety
As noted in the issue:
prompt_cache_keyis a routing hint, not a correctness boundary. A stale or wrong key only causes a cache miss, never a wrong result. This is a low-risk efficiency fix.Backwards Compatibility
prompt_cache_keyset → defaults tosession_id)Test Results
Closes #51395