Skip to content

fix(codex): clamp prompt_cache_key to 64 chars for Codex/ChatGPT backend - #62266

Closed
JoaoMarcos44 wants to merge 1 commit into
NousResearch:mainfrom
JoaoMarcos44:fix/62063-codex-prompt-cache-key-clamp
Closed

JoaoMarcos44 wants to merge 1 commit into
NousResearch:mainfrom
JoaoMarcos44:fix/62063-codex-prompt-cache-key-clamp

Conversation

@JoaoMarcos44

@JoaoMarcos44 JoaoMarcos44 commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Fixes Codex/ChatGPT backend rejects requests when session_id > 64 chars (prompt_cache_key HTTP 400) #62063: the OpenAI Responses / ChatGPT-Codex backend caps prompt_cache_key at 64 chars and returns HTTP 400 ("string too long") past that.
  • agent/transports/codex.py::build_kwargs falls back to the raw session_id for the cache key when there's no static content to hash. session_id is caller-supplied and unbounded (e.g. control-plane session keys like agent:<uuid>:issue:<uuid> run ~140 chars), so any session_id over 64 chars broke every call on this backend.
  • Clamp the resolved cache_key to 64 chars right after it's computed, before it's used as kwargs["prompt_cache_key"] (Codex path) or extra_body["prompt_cache_key"] (xAI path). Per the existing docstring, the key is a routing hint only — never a correctness boundary — so truncation is safe.

Root cause detail

  • _content_cache_key() returns a bounded pck_<hash> (28 chars) whenever instructions or tools is non-empty. In today's call path instructions always falls back to DEFAULT_AGENT_IDENTITY when unset, so in practice the raw-session_id fallback is rarely hit — but it's still reachable (e.g. if that fallback logic changes, or the caller passes tools=None/instructions="" through a path that skips the identity fallback), and when it is hit, the value is completely unbounded. The clamp closes this at the single point of consumption so it can never regress regardless of what feeds cache_key.

Test plan

  • tests/agent/transports/test_codex_transport.py::test_cache_key_session_id_fallback_clamped_to_64_chars — forces the fallback (monkeypatches _content_cache_key to return None) with a 146-char session_id and asserts the resulting prompt_cache_key is exactly 64 chars.
  • tests/agent/transports/test_codex_transport.py::test_content_cache_key_is_none_only_when_both_empty — pins down the exact trigger condition for the fallback.
  • tests/agent/transports/test_codex_transport.py::test_realistic_call_never_leaks_raw_long_session_id_today — production-shaped call (no explicit instructions, matching agent/chat_completion_helpers.py) with a 146-char session_id; confirms the emitted key is always the bounded content hash.
  • Full existing suite for the transport, adapter, and auxiliary client passes unchanged (377 tests).

The OpenAI Responses / ChatGPT-Codex backend rejects prompt_cache_key
values over 64 chars with HTTP 400. The key falls back to raw
session_id when there's no static content to hash, and session_id is
caller-supplied and unbounded (e.g. control-plane session keys like
agent:<uuid>:issue:<uuid> can run ~140 chars), so a long session_id
broke every call. prompt_cache_key is a routing hint only, never a
correctness boundary, so truncating it is safe.

Fixes NousResearch#62063

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/openai OpenAI / Codex Responses API sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) duplicate This issue or pull request already exists labels Jul 10, 2026
@alt-glitch

Copy link
Copy Markdown
Contributor

This was generated by AI during triage.

Duplicate of #24273 (earliest open PR, created 2026-05-12) — same code site and mechanism (bounding prompt_cache_key to 64 chars at the build_kwargs consumption point in agent/transports/codex.py). #24273 uses a SHA-256 hash for oversized keys; this PR truncates to cache_key[:64]. Both fix #62063; the truncation-vs-hash difference doesn't change that they're the same fix at the same branch. Related cluster: #26468, #28307 (cron-side cap), #51037 (closed). A maintainer should pick the canonical implementation.

@falkoro

falkoro commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Verified this against current main and the PR branch:

  • The vulnerable fallback is exactly as described: _content_cache_key(...) or session_id with session_id unbounded, and the clamp sits before both consumers (Codex kwargs and the xAI extra_body path further down).
  • _content_cache_key returns pck_<sha256[:24]> = 28 chars, so the hash path can never hit the cap — the clamp only ever bites the session_id fallback, matching the "defense-in-depth" framing.
  • Ran tests/agent/transports/ on the branch: 352 passed (Linux, Python 3.12), including the three new tests. The third test (test_realistic_call_never_leaks_raw_long_session_id_today) is a nice touch — it documents why production traffic rarely hits the fallback while still pinning the trigger condition.
  • The 64-char backend cap itself I did not exercise against the live API, but it's attested twice independently: the verbatim 400 in Codex/ChatGPT backend rejects requests when session_id > 64 chars (prompt_cache_key HTTP 400) #62063 and salscoding's confirmed-in-production report on fix(codex): bound prompt cache key length #24273.

Prior art note for maintainers: #24273 (May) proposed the same clamp idea (_safe_prompt_cache_key) and deserves credit for the diagnosis, but that branch now diffs at ~4960 files / −1.08M lines against current main and predates the _content_cache_key architecture — this PR is the current, surgical implementation of the fix.

Optional: plain [:64] truncation means two distinct long session keys sharing a 64-char prefix collide into one cache bucket. Harmless (the docstring is explicit that this is a routing hint, never a correctness boundary), but hashing overlong fallbacks in the same pck_ style would preserve distinctness if you ever want it.

LGTM — small, well-tested, and correctly scoped.

@teknium1

Copy link
Copy Markdown
Collaborator

This is an automated hermes-sweeper review. Current main already provides the reported bounded-key behavior, so this duplicate PR can be closed.

The raw fallback expression remains in the source, but current instruction resolution makes it unreachable for the reported API/gateway flow.

@teknium1 teknium1 closed this Jul 11, 2026
@teknium1 teknium1 added the sweeper:implemented-on-main Sweeper: behavior already present on current main label Jul 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint duplicate This issue or pull request already exists P2 Medium — degraded but workaround exists provider/openai OpenAI / Codex Responses API sweeper:implemented-on-main Sweeper: behavior already present on current main sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Codex/ChatGPT backend rejects requests when session_id > 64 chars (prompt_cache_key HTTP 400)

4 participants