investigate(cache): compression-rotation cache-scope gap -- needs maintainer design decision

[JoaoMarcos44]

Summary

Not a fix -- a reproduction for NousResearch/hermes-agent#79017, opened for maintainer input on whether/how to pursue it. Stacked on NousResearch/hermes-agent#78959 (the scope only makes sense once that lands).

_cache_scope_from_session_id() (NousResearch#78959, closing NousResearch#78941) scopes prompt_cache_key by the physical session_id. Correct for isolating unrelated sessions and for cron re-fires of the same job. But context-compression rotation mints a new physical session_id mid-conversation to segment the transcript, so the same logical conversation goes cache-cold at every rotation boundary.

What this PR adds

One xfail(strict=True) test (tests/agent/transports/test_codex_cache_scope_compression_rotation_gap.py) that fails loudly if the behavior is ever silently "fixed" without anyone noticing, and documents the gap in runnable form instead of prose alone.

Why no fix is proposed here

A real fix needs a new concept -- a logical cache-scope (conversation identity) distinct from both the physical session id and provider sticky-routing keys:

Identity	Role	Changes on
Physical session/transcript id	One execution/segment	/new, branch, compression rotation
Logical cache-scope (proposed)	Stable prompt-cache bucket	/new, branch, independent subagent -- not rotation
Provider sticky-routing (OpenRouter session_id, xAI x-grok-conv-id)	Pins requests to the warm backend	Per-provider contract

Threading that scope through the compression/rotation/branch code paths (and NOT reusing gateway_session_key or a generic conversation-root walk, both of which outlive /new or cross branch boundaries incorrectly) is a design task, not a diff this PR wants to force through unreviewed.

Ask

Maintainer call on:

1. Is this worth fixing, given it only costs one cold cache-prefix per rotation (relatively rare)?
2. If yes, does the logical-cache-scope shape above look right before someone implements it?

%%{init: {'theme': 'dark', 'themeVariables': { 'primaryColor': '#8b0000', 'mainBkg': '#0a0204', 'primaryTextColor': '#ffccd5', 'primaryBorderColor': '#ff0038', 'lineColor': '#ff0038'}}}%%
graph TD
    A[Conversation Turn 1<br/>session-root] -->|Compression Threshold Hit| B[Rotation Event]
    B --> C[Conversation Turn 2<br/>session-rotated]

    A -->|scope = session-root| D[Cache Bucket: pck_root_hash]
    C -->|scope = session-rotated| E[Cache Bucket: pck_rotated_hash]

    D -. Same Logical Conversation .-> E
    D -->|Rotation Boundary| F[Cold Cache: Warm Prefix Lost]
    E --> F

    style F fill:#8b0000,stroke:#ff0038,color:#ffccd5
    style D fill:#3a0000,stroke:#ff0038,color:#ffccd5
    style E fill:#3a0000,stroke:#ff0038,color:#ffccd5

Loading

Test plan

• pytest tests/agent/transports/test_codex_cache_scope_compression_rotation_gap.py -v -> 1 xfailed (expected; strict, will error if it starts passing without a matching design)
• No production code touched -- test-only change, zero risk to fix(cache): scope prompt_cache_key by session to stop cross-session bucket sharing NousResearch/hermes-agent#78959 / [Cache]: content-only prompt_cache_key can concentrate unrelated sessions into one routing scope NousResearch/hermes-agent#78941

Discussion for NousResearch#79017 (design decision, not code fix).

[JoaoMarcos44]

Superseded by NousResearch#79036 -- recreated standalone/self-contained so it doesn't drag in NousResearch#78959's unmerged diff. Closing.