fix(cache): honor the host-declared conversation key on the affinity-key path

[JoaoMarcos44]

What does this PR do?

Lets the host declare the logical conversation behind its physical session ids, and makes the four conversation-affinity surfaces consult that declaration. This is the fix for the key churn demonstrated in #96811 — the mechanism behind the cost half of #96570.

Root cause

Every affinity hint Hermes sends is derived from the physical session id:

Consumer	Wire field	Derived from
agent/transports/codex.py, agent/transports/chat_completions.py	prompt_cache_key	resolve_prompt_cache_scope() → compression-lineage root → physical id
plugins/model-providers/openrouter	body.session_id (sticky routing)	ambient conversation id → get_conversation_root() → physical id
plugins/model-providers/nous	body.session_id (sticky routing)	same
plugins/model-providers/openrouter	x-grok-conv-id header	same

A host that mints one physical session per response therefore re-keys all four on every reply, and the conversation never lands back on the routing bucket it just warmed. Two hosts do exactly that:

1. Hermes Studio group chat — groupRuntimeSessionId() mints gc_run_<room>_<profile>_<name>_<uuid4hex> per reply and destroys the session afterwards.
2. POST /v1/responses with client-managed history — session_id = stored_session_id or str(uuid.uuid4()) in gateway/platforms/api_server.py. Without previous_response_id that is a fresh id per request, even though the same handler parsed X-Hermes-Session-Key a few lines earlier and passes it straight to _create_agent(gateway_session_key=…).

Case 2 closes the whole loop inside this repo: the stable identity is already in hand, already on the agent — the affinity path just never consulted it.

Why not normalize the id

Stripping a per-run-looking suffix is not identity-safe, and #96768 already carries both negative controls as tests: POST /v1/sessions preserves a client-supplied id verbatim (two independent conversations may differ only in a trailing 32-hex identity), and Studio truncates its semantic prefix to 96 characters before appending the token (two members of one room can end up distinguished by that token alone). Same failure class as #79017: syntactic inference truncates legitimate identities and collides distinct owners.

The declaration is semantic instead, and the carrier already exists: gateway_session_key, the "stable per-chat key" (agent:main:telegram:dm:123) built by gateway.session.build_session_key() from X-Hermes-Session-Key. Its own documentation notes that branching keys off session_id, not this slot — exactly the property a conversation scope needs. agent/prompt_cache_scope.py simply did not read it. (AIAgent.__init__ already accepts it, and run_agent.py:741 already treats it as the context engine's conversation_id.)

Changes

• agent/prompt_cache_scope.py — declared_conversation_scope() resolves the key to gwk_<sha256[:24]> and outranks the lineage walk: it is stable across compression rotation and across per-response ids, which the walk cannot see. Hashed because — unlike a session id — the key embeds platform/chat/user identifiers and leaves the process verbatim as OpenRouter's sticky session_id and as x-grok-conv-id; hashing also keeps it inside every consumer's length budget (28 chars).
• agent/portal_tags.py — a second ambient scope for routing, kept separate from the conversation id, which is an attribution value (conversation=<id> on Portal requests). The two agree for any host that keeps one id per conversation; they diverge only for a per-response host. It is published only when a host declared something, so unset ⇒ providers fall back to the attribution id exactly as before: delegate trees keep sharing their parent's sticky key and Portal attribution is untouched.
• hermes_state.py — is_explicit_fork_child(), the public view of _is_explicit_fork_child_row()'s marker rules, so the fork boundary is enforced without re-implementing it.
• run_agent.py — publishes the declared routing scope beside the existing conversation context (turn entry, and the out-of-turn compaction forwarder), with symmetric resets.
• openrouter / nous plugins — get_affinity_scope() or get_conversation_context() or session_id.

Isolation the declaration must not cross (#79161)

/branch children, delegate subagents and tool children share their parent's chat key but are separate conversations — the session row's fork markers keep them on their own scope. Background-review forks clone the live runtime (key included), so _persist_disabled excludes them. A DB error during that check degrades to the physical-id scope rather than risking a merge.

Blast radius

Zero wire change for any agent without a declared key: no key ⇒ declared_conversation_scope() returns None and every path resolves exactly as on main. POST /v1/responses + X-Hermes-Session-Key is fixed with no host change. Studio group chat needs one kwarg on its side (AIAgent(..., gateway_session_key=<stable per-room-member key>)), since its bridge constructs the agent directly; its truncate-before-nonce ordering is a separate producer-side bug worth fixing there too.

%%{init: {'theme': 'dark', 'themeVariables': { 'primaryColor': '#8b0000', 'mainBkg': '#0a0204', 'primaryTextColor': '#ffccd5', 'primaryBorderColor': '#ff0038', 'lineColor': '#ff0038'}}}%%
graph TD
    A[Reply N - gc_run_room_member_uuidA] --> R{Scope Resolver}
    B[Reply N+1 - gc_run_room_member_uuidB] --> R

    R -->|Declared chat key present| K[Hashed Logical Scope - gwk_sha256]
    R -->|No declaration| P[Physical Id / Lineage Root - unchanged]
    R -->|Branch, Delegate, Tool child| F[Own Scope - fork isolation kept]
    R -->|Background review fork| F

    K --> S1[prompt_cache_key]
    K --> S2[OpenRouter sticky session_id]
    K --> S3[Nous Portal sticky key]
    K --> S4[x-grok-conv-id]

    S1 --> W[One Warm Routing Bucket across every reply]
    S2 --> W
    S3 --> W
    S4 --> W

Loading

Related Issue

Fixes #96811
fixes #96570

Type of Change

• Bug fix (non-breaking change which fixes an issue)

How to Test

scripts/run_tests.sh tests/agent/test_declared_conversation_scope.py tests/agent/test_prompt_cache_scope.py tests/agent/test_portal_tags.py tests/providers tests/agent/transports tests/hermes_state -q

Local results: 22 passed (new file), 433 passed / 1 failed across the adjacent suites, 200 passed (tests/hermes_state). The single failure — tests/agent/test_portal_tags.py::test_compress_context_preserves_ambient_context — reproduces identically on unmodified main in this environment (AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer', a Python 3.14 concurrent.futures change surfacing in tools/daemon_pool.py), so it is not caused by this branch. ruff check clean on all changed files.

New coverage in tests/agent/test_declared_conversation_scope.py: two per-response ids resolving to one scope; distinct declarations staying isolated; the raw key never appearing in the scope; no-declaration behavior identical to the lineage path; /branch, delegate, tool and background-review forks ignoring the declaration; the fork-check DB failure degrading safely; prompt_cache_key stability on both transports; sticky session_id and x-grok-conv-id following the declared scope while an undeclared turn still resolves through the conversation id.

[cervantesh]

Thanks for isolating a real Hermes-native reproduction for the affinity-key churn. The /v1/responses path can mint a fresh physical UUID for each client-managed-history request while X-Hermes-Session-Key is already available to the agent. The write-loss premise from #96570 remains disproved, but this separate mechanism is valid and belongs to #96811.

I reviewed exact PR head 0c5929f68a against current main@0abecf7a93 and also applied the PR cleanly onto that main revision. The candidate demonstrates the intended improvement, but I found two correctness blockers. The earlier real-entry-point evidence gap is now closed by the reproduction package linked below.

1. gateway_session_key has a longer lifetime than a conversation

The proposed precedence treats gateway_session_key as a host-declared logical-conversation identity. That conflicts with its existing documented contract.

PR #20199 introduced X-Hermes-Session-Key specifically as a stable per-channel, long-term-memory scope independent of the transcript ID: one stable key per assistant channel across transcripts that rotate on /new. The current API-server documentation says the same thing: gateway_session_key persists across transcripts while session_id rotates on /new. Native messaging gateways also pass ctx.session_key into every newly created agent, so this is not limited to unusual /v1/responses clients.

That creates the following reachable lifecycle:

1. conversation A runs under physical session ID A and gateway key K;
2. /new calls reset_session(K) and creates physical session ID B;
3. the next agent is created with the same gateway key K; and
4. declared-key precedence maps both A and B to the same gwk_<sha256(K)>.

I first confirmed that behavior at the resolver level. I then drove both production boundaries through a real-path differential:

• two actual POST /v1/responses requests carrying one X-Hermes-Session-Key; and
• the native gateway's actual /new handler backed by a temporary real SessionStore and SessionDB.

The effective result was:

Boundary	current main	PR head	current main + PR
Consecutive Responses requests keep all affinity values	no	yes	yes
/new rotates all affinity values	yes	no	no

For consecutive Responses requests, the PR correctly kept all six values stable:

• Chat Completions prompt_cache_key;
• Responses API prompt_cache_key;
• OpenRouter sticky session_id;
• Nous sticky session_id;
• x-grok-conv-id; and
• the internal resolved scope.

That is a direct positive witness for the #96811 fix. However, the same six values also remained identical across /new, even though the physical session ID rotated. This conflicts with the cache-scope contract established in #79017 and implemented by #86733: continuity survives compression rotation, but the scope resets on /new, /branch, and other independent-conversation boundaries.

This also makes the blast radius broader than the PR description suggests. Every ordinary native gateway supplies this key, so normal messaging conversations would move from physical/lineage affinity to per-chat affinity—not only hosts that mint one ID per response.

There is related work around the /v1/responses conversation identity in #47294, #41209, and #16517. Those are not duplicates of #96811: they cover namespacing, ownership, and durable continuity rather than provider affinity, but they may contain a semantic owner closer to a logical conversation than the deliberately cross-transcript gateway key. Conversely, #71556/#71608 use gateway_session_key for Langfuse grouping, where per-channel observability is a different and intentionally broader contract.

This is not future hardening beyond #96811. It is a negative control on the lifetime of the identity carrier introduced by this PR. A safe contract would need to remain stable across per-response physical IDs while rotating at /new and preserving the existing branch/delegate isolation.

2. The current head has a reproducible cleanup regression

The hosted Python test job is red, and I reproduced the affected selection locally:

4 failed, 9 passed

All four failures reach the outer cleanup before affinity_token has been assigned. The variable is created only after durable-lease, Relay, and task-start operations that may return or raise, but the outer finally reads it unconditionally at run_agent.py:9092, producing:

UnboundLocalError: cannot access local variable 'affinity_token'
where it is not associated with a value

This is attributable to the candidate change rather than CI infrastructure.

For balance, the new focused suite passes locally:

tests/agent/test_declared_conversation_scope.py: 22 passed

The adjacent prompt-cache, Portal-tag, and API-server selections also pass:

141 passed

Those green results validate the added helper and provider wiring; they do not negate the early-exit regression.

Real-path reproduction package

The complete witness, exact revisions, summarized outputs, instructions, and integrity hash are here:

https://gist.github.com/cervantesh/1fa5099000d0163478d5fb6a6a325b47

It makes zero model-provider network calls. It uses the real Responses handler and real /new handler, replacing only model execution with a deterministic capture. The resulting identities are then passed through the repository's actual resolver, OpenAI-wire transports, and OpenRouter/Nous provider profiles. The result was repeated and remained the same after integrating the PR onto current main.

Assessment

My current assessment is REQUEST CHANGES, for two strict reasons:

• the selected identity demonstrably crosses /new, violating an existing isolation invariant; and
• the exact PR head has a reproducible cleanup regression.

The general direction—letting a host declare a semantic affinity owner instead of inferring identity from UUID syntax—still looks sound. Evidence that would change this assessment would be:

• a declaration or derivation whose lifecycle is explicitly per logical conversation rather than per channel;
• a negative control proving /new receives a distinct scope;
• the real-path witness remaining GREEN for consecutive per-response requests; and
• green required checks on the resulting exact head.

Studio can remain a separately acknowledged integration step because it still needs to supply whichever per-conversation declaration is adopted.

[StanleyStetson]

Following the review feedback on #98170 and the discussion on #97158 / #97709, I put together an exploratory revision branch built on top of this salvage (preserving @JoaoMarcos44's and @kshitijk4poor's commits via cherry-pick).

Sharing here in case any of these pieces are helpful for the salvage effort:

1. Memoization & Hot Path: Kept declared_conversation_scope() strictly inside the memoization miss path in resolve_prompt_cache_scope(), avoiding SQLite queries on cached calls. The memo key is (sid, db_present, epoch).
2. Epoch Layer: Mixed conversation_epoch into the declared scope hash (key:epoch for epoch > 1) and wired it through SessionStore.reset_session() and TurnContext to handle /new rotation while keeping affinity stable across per-response requests.
3. Tests: Added tests for epoch rotation, memo invalidation on epoch advance, and ABA monotonicity under mixed /new + auto-resets.

The branch and commit are available here if you'd like to inspect, cherry-pick, or adapt anything from it:

• Branch: StanleyStetson:salvage/96811-epoch-revision
• Commit: 4a6db3cda665aff33d87e07912bf2d3fa15af375

[kshitijk4poor]

Closing this revision while keeping the bug (#96811) open — the mechanism is real and your reproduction package proved it, but the carrier as implemented has a verified lifecycle violation that can't be fixed by a follow-up commit on this branch.

State of the review, consolidated:

• Confirmed real: per-response hosts (Studio group chat, /v1/responses client-managed history) re-key all four affinity surfaces every reply; this PR correctly stabilizes all six wire values across consecutive Responses requests (cervantesh's differential, reproduced independently).
• Confirmed blocker: gateway_session_key has a longer lifetime than a conversation. reset_session() (/new) mints a fresh physical id but keeps the chat key, so declared-key precedence maps pre- and post-/new conversations onto one gwk_ scope — violating the prompt_cache_key loses continuity across context-compression session rotation (needs a logical cache-scope concept) #79017/fix(cache): keep prompt_cache_key warm across compression session rotation #86733 rotation contract for every native messaging gateway, not just per-response hosts. Reproduced on the PR head with a real SessionDB (see the note on fix(cache): honor the host-declared conversation key on the affinity-key path (salvage of #97158) #97709).
• Why not a follow-up commit: a correct carrier must be stable across per-response physical ids within one logical conversation AND rotate on /new. The chat key alone cannot express that; it needs a per-conversation epoch minted at reset_session()/conversation creation and carried alongside the key — a change to the declaration contract itself, including coverage for the /v1/responses and Studio boundaries where no gateway SessionStore epoch exists. (fix(cache): honor host-declared session key and conversation epoch for prompt cache affinity #98170 attempted the epoch but wired it only through the native gateway and put a DB probe on the per-call hot path; closed as well.)

Your diagnosis, the four-surface consumer map, the gwk_ hashing scheme, and the fork/background-review exclusions all carry into the revision, and the salvage vehicle (#97709) preserves your commit and authorship — the revised contract will land with your credit. Thanks for a genuinely rigorous PR; this one moved the design forward even though this revision can't merge.