fix(cache): mix conversation epoch into declared scope for /new rotation (#96811)

[StanleyStetson]

Summary

Salvage and revision of #97158 and #97709 by @JoaoMarcos44 and @kshitijk4poor onto current main, preserving original authorship via cherry-pick.

Resolves #96811: hosts that mint per-response session IDs (Hermes Studio group chat, POST /v1/responses with client-managed history) re-keyed all conversation affinity surfaces on every reply (prompt_cache_key, OpenRouter/Nous sticky session_id, and x-grok-conv-id).

This PR implements the full key + conversation_epoch contract requested in the reviews of #97158 and #97709:

1. Declared Scope: Derives gwk_<sha256(key:epoch)[:24]> from gateway_session_key (X-Hermes-Session-Key / build_session_key), keeping per-response replies on the warmed cache bucket.
2. Lifecycle & /new Rotation: Mixes conversation_epoch into the hash (advancing on explicit /new in SessionStore.reset_session() and monotonically on idle/daily policy auto-resets in get_or_create_session), ensuring that post-reset conversations rotate to fresh provider affinity buckets.
3. Fork Isolation: Explicit fork children (/branch, delegate subagents, tool-spawned sessions) and background review forks (_persist_disabled) bypass the declared key and retain their isolated physical scope.
4. Hot-Path Memoization: Evaluates declared_conversation_scope() strictly inside the memoization miss path of resolve_prompt_cache_scope(), ensuring zero SQLite queries on subsequent calls within the turn.
5. Blast Radius & Continuity: Hosts without a declared key (direct CLI, standalone desktop) retain exact byte-identical physical/lineage scoping. Gateway conversations carrying a declared key seamlessly transition to the canonical gwk_<hash> scope on deployment.

Changes

• agent/portal_tags.py: Ambient affinity scope ContextVar, set per turn and compacted session.
• agent/prompt_cache_scope.py: declared_conversation_scope() mixing key:epoch into gwk_, memoized under (session_id, db_present, epoch).
• gateway/session.py: conversation_epoch field on SessionEntry, get_conversation_epoch() on SessionStore, incremented on reset_session() and monotonic auto-resets.
• gateway/turn_context.py: Added conversation_epoch to TurnContext.
• agent/agent_init.py & run_agent.py: Plumbed gateway_conversation_epoch through init_agent to agent._gateway_conversation_epoch.
• gateway/run.py: Populated conversation_epoch in TurnContext and passed to AIAgent.
• hermes_state.py: Public is_explicit_fork_child(session_id) helper.
• tests/agent/test_declared_conversation_scope.py: 31 tests covering stability across per-response IDs, epoch advance on /new, memo invalidation, ABA monotonicity, and disk persistence reload.

Validation

• Ran full test suite: 64/64 tests passed across test_declared_conversation_scope.py (31 passed) and test_prompt_cache_scope.py (33 passed).
• Ran lease and session lifecycle suites: 19/19 tests passed green in test_turn_lease.py and test_durable_turn_lease.py.
• Verified hot-path memoization via spy on SessionDB.get_session (0 additional queries on repeated calls).

Closes #96811. Refs #96570, #97158, #97709.

[teknium1]

Review of head 4a6db3cda665aff33d87e07912bf2d3fa15af375

Reviewed against current origin/main (d63f996). Premise verified: /v1/responses with client-managed history mints str(uuid4()) per request, api_server keys _last_resolved_model off gateway_session_key for exactly this reason (api_server.py:3069-3075), and on main resolve_prompt_cache_scope has no way to see a host-declared conversation — so all four affinity surfaces (prompt_cache_key on both wire transports, OpenRouter/Nous sticky session_id, x-grok-conv-id) churn per reply. The bug is real and this PR's core mechanism (declared gwk_<sha256(key[:epoch])[:24]> scope, contextvar publication in portal_tags, precedence over the lineage walk) is sound and well-commented.

Ran the PR's test files in an isolated worktree under a memory cap: tests/agent/test_declared_conversation_scope.py 31/31 passed, tests/agent/test_prompt_cache_scope.py 33/33 passed. Test quality is good — they assert real wire-layer outputs (build_kwargs()["prompt_cache_key"], build_extra_body()["session_id"], extra_headers["x-grok-conv-id"]), fork/_persist_disabled boundaries, memo invalidation, and a live AIAgent.run_conversation contextvar shadowing test — not restatements of the gate expression.

What holds up:

1. Fork isolation — declared_conversation_scope() (agent/prompt_cache_scope.py:139-177) correctly bails on _persist_disabled, on is_explicit_fork_child, and degrades to physical scope on DB error rather than risking a fork merging onto its parent's key. hermes_state.is_explicit_fork_child is a clean read-only wrapper over the existing row-marker rules.
2. Memoization — declared_conversation_scope() runs only inside the miss path of resolve_prompt_cache_scope(); memo key now (sid, db_present, epoch) so /new invalidates. After the first resolution per segment there are zero SQLite queries per turn. Claim verified in code.
3. No-declaration hosts — CLI/desktop paths never set _gateway_session_key; set_affinity_scope(None) is a published-None, and all three provider call sites fall back to get_conversation_context() or session_id — byte-identical to pre-PR behavior. Confirmed by test test_without_a_declaration_the_conversation_id_still_wins.
4. run_conversation early-return safety — affinity_token = None initialized alongside token before the turn-lease early returns (run_agent.py, the #97158 UnboundLocalError class); compression path only sets the scope when unset.

Blockers

1. Epoch is not durable — retired-scope reuse (ABA) through prune/recovery/restart. conversation_epoch lives only on SessionEntry in sessions.json. Three paths roll it back to 1 and re-issue a retired affinity identity:

   • SessionStore.prune_old_entries (gateway/session.py:~3394) pops the entry; the next get_or_create_session for that key creates a fresh entry with conversation_epoch=1, whose hash (epoch-1 omits the :epoch suffix) equals the ORIGINAL epoch-1 conversation's gwk_ scope — a scope retired at the first /new.
   • The recovery path (_recover_session_for_peer reopening agent_close/ws_orphan_reap rows) rebuilds entries from state.db, which stores no epoch — recovered entries come back at epoch 1.
   • Any sessions.json loss/corruption resets every chat's epoch while state.db transcripts survive.
     The prior review rounds on #97158/#97709 demanded a generation that provably never reuses a retired value; this implementation cannot prove non-reuse. test_mixed_new_and_auto_reset_never_rolls_back_epoch_aba only covers in-memory monotonicity within one live entry — none of the three paths above are tested. (Competing #98811 keeps the generation in a durable conversation_generations counter in state.db advanced transactionally with each boundary, which is the correct shape for this.)

2. No rotation on the host the issue is about. The epoch plumbing (TurnContext.conversation_epoch, gateway/run.py:5858/5912/29549) only reaches native gateway platforms — which keep one session_id per chat and were never churning. gateway/platforms/api_server.py (the /v1/responses + Studio group-chat host, i.e. the actual #96811 reproducer) constructs AIAgent with gateway_session_key but no epoch source; _gateway_conversation_epoch stays 1 forever. Result: per-response churn is fixed there (good), but a client starting a NEW conversation on the same X-Hermes-Session-Key stays pinned warm to the previous conversation's affinity bucket indefinitely — violating the cold-across-conversation half of the contract precisely where the bug manifests. There is no api_server test in this PR.

3. api_server identity fragmentation untouched. This PR stabilizes the affinity hash only; the per-request session rows themselves remain unlinked (no session_key bind, no reset-fenced reuse), so transcript/identity fragmentation on the api_server path persists. #98811 carries that repair (_declared_conversation_session / _bind_declared_conversation with reset-fenced recovery mirroring SessionStore._recover_session_for_peer); #98795 has no equivalent.

Nits

4. gateway/run.py:29549-29558 inlines the epoch lookup (with a _entries reach-in fallback) into the ~29k-line turn-assembly path. If the epoch mechanism survives, this belongs on SessionStore only (get_conversation_epoch already exists — the private-dict fallback duplicates it) and adds authority to a god-file the competing design avoids entirely.
5. Deploy-time one-time scope flip: on upgrade, every in-flight native-gateway conversation moves from physical-id scope to gwk_ — one full prompt-cache invalidation per active conversation. Between conversations this is fine; worth stating in the PR body as an accepted cost.
6. Fork-child pre-row window: test_declaration_applies_before_the_row_lands asserts the declared scope applies before _ensure_db_session persists the row. For a fork child whose row hasn't landed yet, is_explicit_fork_child returns False (missing row ≠ fork) and the parent's declared scope is memoized for the segment (root is not None → memoized). Low likelihood today (branch/delegate rows are created eagerly, delegates don't carry the key), but nothing pins it.

Overlap with #98811

Same issue, same lineage (#97158 → #97709), directly conflicting hunks in agent/portal_tags.py, agent/prompt_cache_scope.py, hermes_state.py, both provider plugins, run_agent.py, and the same test filename — only one can land. #98811's mechanism (durable boundary counter in state.db + api_server-layer session rebinding) addresses blockers 1–3 above by construction, has been through four review rounds with blockers addressed, and confines its changes to prompt_cache_scope/api_server/hermes_state without adding plumbing to gateway/run.py, gateway/session.py, turn_context.py, or the init_agent signature. #98795's contextvar + run_conversation publication is the cleaner way to reach the aux-call sites, but its generation mechanism is the weaker of the two.

Verdict: REQUEST CHANGES. The declared-scope core is solid and the tests are real, but the epoch is not durable (blocker 1), never rotates on the api_server host the issue is about (blocker 2), and the identity repair is missing (blocker 3). As written, #98811 is the stronger landing vehicle for the generation scheme; if the maintainers prefer this PR's plumbing style, it needs a durable generation source and api_server coverage — at which point it converges on #98811's design.

[StanleyStetson]

Thank you to everyone who reviewed this PR. I've been following the discussion on #98811 closely, and after reading the maintainer review there, I'm convinced @JoaoMarcos44's approach is the stronger one to carry this fix to production.

#98811 not only addresses the same issue (#96811) but resolves it from a place that better survives the real failure modes — a durable generation counter in the database rather than per-chat state in sessions.json, plus the API-server identity repair that this PR doesn't touch. It has been through several thorough review rounds, all blocking findings are addressed, and the maintainer review returned no blocking issues.

Since both PRs solve the same problem and overlap across the same files, only one of them should merge. I'm closing this one in favor of #98811.

I'm grateful for the time and review feedback here — it helped me understand the tradeoffs much better. Let me know if I can help in any way with #98811.