Skip to content

fix(codex): bound prompt cache keys - #26468

Closed
JackALaing wants to merge 1 commit into
NousResearch:mainfrom
JackALaing:fix/codex-prompt-cache-key-length
Closed

fix(codex): bound prompt cache keys#26468
JackALaing wants to merge 1 commit into
NousResearch:mainfrom
JackALaing:fix/codex-prompt-cache-key-length

Conversation

@JackALaing

Copy link
Copy Markdown

Summary

  • Keep OpenAI Responses prompt_cache_key values within the provider's 64-character limit.
  • Preserve existing short session IDs unchanged for prompt-cache continuity.
  • Use a stable hash suffix for longer session IDs, including cron session IDs derived from long job names.

Test Plan

  • python -m pytest tests/agent/transports/test_codex_transport.py -q
  • python -m py_compile agent/transports/codex.py
  • git diff --check

Platforms Tested

  • macOS arm64, Python 3.13.13.

Related / competing PRs

  • PR #10006 improves Codex prompt-cache routing headers and usage accounting, but does not touch agent/transports/codex.py or bound the OpenAI prompt_cache_key body field.
  • PR #5252 proposes broader provider-aware session-affinity routing. This PR is a smaller bug fix for the current Responses transport and can stand independently.
  • PR #6461 was a closed SSE proxy cache-routing attempt; it does not supersede this OpenAI request validation fix.
  • Searched PRs/issues for prompt_cache_key length, prompt_cache_key Responses, and prompt_cache_key; found no open or merged PR specifically fixing the 64-character limit failure.

Notes/Risks

  • Long cache keys remain deterministic, so repeated calls for the same Hermes session still share the same prompt-cache identity.
  • Short cache keys are unchanged to avoid unnecessary cache churn.
  • Cross-platform impact is low: this only changes request kwargs construction for the Responses API transport and adds no file I/O, shell commands, process management, or platform-specific behavior.

@alt-glitch

Copy link
Copy Markdown
Collaborator

Likely duplicate of #24273 — both fix Codex transport prompt_cache_key exceeding OpenAI's 64-character limit by hashing oversized session IDs. Same bug (400 errors on long cron session IDs), same fix approach.

@alt-glitch alt-glitch added P2 Medium — degraded but workaround exists type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/openai OpenAI / Codex Responses API labels May 15, 2026
@JackALaing

Copy link
Copy Markdown
Author

The main difference here is that this PR preserves a readable prefix and appends a short hash, rather than replacing the entire value with a full SHA256 digest. That keeps the key within OpenAI’s 64-char limit while preserving some operator/debug value in logs and traces. For example, a long cron/session ID still shows the job/session family prefix, with the hash suffix preventing collisions.

I’m fine closing this in favor of #24273 if maintainers prefer the older PR.

@teknium1

Copy link
Copy Markdown
Contributor

Thanks for the focused cache-key fix. This is an automated hermes-sweeper review: current main already provides the reported bounded-key behavior through a stronger content-addressed approach.

The current implementation both stays within the provider limit and avoids cache-cold recurring cron runs.

@teknium1 teknium1 closed this Jul 13, 2026
@teknium1 teknium1 added sweeper:implemented-on-main Sweeper: behavior already present on current main sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists provider/openai OpenAI / Codex Responses API sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:implemented-on-main Sweeper: behavior already present on current main sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants