Skip to content

fix(codex): bound prompt cache key length - #24273

Closed
Armib20 wants to merge 1 commit into
NousResearch:mainfrom
Armib20:ali/fix-codex-prompt-cache-key
Closed

fix(codex): bound prompt cache key length#24273
Armib20 wants to merge 1 commit into
NousResearch:mainfrom
Armib20:ali/fix-codex-prompt-cache-key

Conversation

@Armib20

@Armib20 Armib20 commented May 12, 2026

Copy link
Copy Markdown

Summary

  • Bounds Codex prompt_cache_key to the backend's 64-character limit.
  • Preserves short session IDs unchanged for debuggability.
  • Hashes oversized session IDs to a stable SHA-256 hex key.

Why

Hermes session IDs can exceed the Codex backend prompt_cache_key maximum, causing non-retryable 400 errors:

Invalid 'prompt_cache_key': string too long. Expected a string with maximum length 64

Test Plan

  • ~/.hermes/hermes-agent/venv/bin/python -m pytest tests/agent/transports/test_codex_transport.py tests/run_agent/test_run_agent_codex_responses.py -q

@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/openai OpenAI / Codex Responses API P2 Medium — degraded but workaround exists labels May 12, 2026
@salscoding

Copy link
Copy Markdown

Confirming this in production — external client path, not just internal session_id

Most reports here cover the internal path where Hermes generates the session_id and passes it as prompt_cache_key. We hit the same 400 from the external client path: Paperclip's hermes_gateway adapter sends a client-supplied prompt_cache_key to Hermes's OpenAI-compatible API server (/v1/chat/completions), and Hermes forwards it unchanged to the Codex transport — where it blows the 64-char limit.

Error:

HTTP 400: Invalid 'prompt_cache_key': string too long. Expected a string with maximum length 64, but got a string with length 138 instead.

Setup: self-hosted Paperclip → self-hosted Hermes Agent (Docker, http://hermes:8642) via hermes_gateway adapter. Paperclip's "Issue scoped" session key strategy generates a paperclip:company:<uuid>:agent:<uuid>:issue:<uuid> key (138 chars) and sends it as prompt_cache_key in the request body. Hermes passes it through unchanged to OpenAI, which rejects it.

The root fix is in Paperclip (paperclipai/paperclip#8714 hashes over-length keys before sending). But this PR would add a meaningful second defensive layer in the Codex transport — clamping any over-length prompt_cache_key regardless of whether it was generated internally by Hermes or supplied by an external client through Hermes's API server. That makes Hermes robust against any upstream client that sends a long key.

Supporting this PR.

@teknium1 teknium1 added sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 13, 2026

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the Codex cache-key failure mode. The session-ID path this patch changes has since been replaced on main by content-addressed cache keys: agent/transports/codex.py:16-46 creates a pck_ + 24-character digest, and build_kwargs() uses it at agent/transports/codex.py:252-263 (commit 0a741ed622c65b8c1a304c46fd489e03c171052d).

Problems

  • The proposed guard only wraps session_id. Current main applies request_overrides after its generated key at agent/transports/codex.py:295-297, so an explicit top-level prompt_cache_key can still replace it unbounded.
  • The xAI body path also preserves a caller-provided cache key via setdefault at agent/transports/codex.py:370-375; tests/agent/transports/test_codex_transport.py:150-165 explicitly verifies that preservation.

Suggested changes

  • Rework the patch around the current content-addressed implementation, then validate the final effective top-level and xAI body override values with focused long-key tests.

Automated hermes-sweeper review.

Comment thread agent/transports/codex.py
@@ -101,7 +120,7 @@ def build_kwargs(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This only bounds the session_id-derived value. On current main, request overrides are merged afterward (agent/transports/codex.py:295-297), so an explicit prompt_cache_key can still replace it unbounded; the xAI extra_body override is similarly preserved. Please rework this around the final merged values if the goal includes caller-provided keys.

@teknium1

Copy link
Copy Markdown
Contributor

Thanks @Armib20 — you filed the earliest fix for the over-length Codex prompt_cache_key (>64 → HTTP 400) bug. Resolved on main in #66330 (commit 61bbc39), which bounds the cache-scope headers plus the top-level and xAI extra_body override keys and reapplies the guard at final preflight. Your original diagnosis is credited in the AUTHOR_MAP and PR body. Closing as implemented.

@teknium1 teknium1 closed this Jul 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists provider/openai OpenAI / Codex Responses API sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants