feat(router): coordinate session affinity across replicas [DYN-3249]

components/src/dynamo/common/configuration/groups/router_args.py

@@ -244,9 +244,11 @@ def add_arguments(self, parser) -> None:
             env_var="DYN_ROUTER_SESSION_AFFINITY_TTL_SECS",
             default=None,
             help=(
-                "Enable router-local session affinity with this idle TTL in seconds. "
-                "Affinity is disabled when this option is omitted. "
-                "This is independent of KV prediction TTL settings."
+                "Enable session affinity and set the process-local cache eviction TTL "
+                "in seconds. etcd and shared FileStore use immutable distributed claims "
+                "whose lifetime follows the creating frontend, not this TTL. Memory and "
+                "Kubernetes discovery remain process-local. Affinity is disabled when "
+                "this option is omitted."
             ),
             arg_type=int,
             dest="session_affinity_ttl_secs",

docs/agents/session-ids.md

@@ -7,7 +7,11 @@ subtitle: Identify agent sessions from supported coding agents and custom client
 
 A session ID is the stable identifier Dynamo uses for one agent reasoning/tool chain. A root agent, planner, researcher subagent, or OpenCode subtask can each have its own session. Every LLM request in that chain should carry the same `session_id`; child sessions can also carry a `parent_session_id` so traces and replay tools can rebuild the tree. Some academic papers also call this a `program_id`.
 
-Session identity is passive metadata. Sending `X-Dynamo-Session-ID` does not enable sticky sessions or change request placement. Tracing records the identity when `DYN_REQUEST_TRACE` is enabled, and a session-aware routing policy can consume it only when that policy is configured separately.
+Session identity is passive metadata unless session affinity is explicitly enabled.
+Sending `X-Dynamo-Session-ID` alone does not change request placement. Tracing records
+the identity when `DYN_REQUEST_TRACE` is enabled. When
+`--router-session-affinity-ttl-secs` is configured, the router uses the ID for an
+immutable endpoint- and phase-scoped worker binding.
 
 ## Session ID Inputs
 
@@ -29,7 +33,18 @@ Dynamo also recognizes the current stable identity headers emitted by the follow
 | Codex | `session-id` | None | `session-id` becomes the `session_id`. |
 | OpenCode | `x-session-id` | `x-parent-session-id` | `x-session-id` becomes the `session_id`; `x-parent-session-id` becomes `parent_session_id` when present. |
 
-`X-Dynamo-Session-Final` applies with either canonical or agent-native session identity.
+`X-Dynamo-Session-Final` applies with either canonical or agent-native session
+identity. With session affinity enabled, a final request routes normally and then
+terminally closes its binding. Close invalidation across replicas is eventual. Do not
+send more requests with that session ID after close.
+
+etcd and FileStore on a shared filesystem coordinate bindings across frontend
+processes. MemoryStore and Kubernetes discovery retain process-local affinity only.
+The affinity TTL controls local cache cleanup, not the distributed claim lifetime.
+Claims follow the creating frontend's etcd lease or FileStore ownership. If a claim
+expires or its bound worker disappears, create a new session with a new ID instead of
+reusing or rebinding the old ID. See [Router session affinity](../components/router/router-configuration.md#session-affinity)
+for the full contract.
 
 ### Custom Agent Harnesses
 

docs/components/frontend/nvext.md

@@ -54,7 +54,7 @@ token IDs, pass integer IDs in the normal `stop` array, for example
 `"stop": [576]`. Strings such as `"token_id:576"` remain literal string stop
 sequences and are not parsed as token IDs.
 
-### Header Overrides
+### Header overrides
 
 Routing fields can also be set via HTTP headers, which take priority over `nvext` values:
 
@@ -75,9 +75,13 @@ session headers described in [Session IDs](../../agents/session-ids.md);
 `nvext` does not accept session identity fields.
 
 When session affinity is enabled with `--router-session-affinity-ttl-secs`, the
-router also uses `X-Dynamo-Session-ID` for router-local affinity. See
-[Configuration and Tuning](../router/router-configuration.md#session-affinity)
-for routing behavior and TTL settings.
+router uses `X-Dynamo-Session-ID` for immutable endpoint- and phase-scoped affinity.
+On etcd and shared FileStore, replicas coordinate through a distributed claim while
+the request hot path uses a process-local cache. Existing local or shared bindings
+override routing headers; the headers above are proposals only when no binding exists.
+Memory and Kubernetes discovery do not provide cross-process affinity. See
+[Configuration and Tuning](../router/router-configuration.md#session-affinity) for
+claim lifetime, cache TTL, terminal close, and failure behavior.
 
 For trace sink configuration and JSONL schema details, see
 [Agent Tracing](../../agents/agent-tracing.md).

docs/components/router/router-configuration.md

@@ -111,31 +111,43 @@ a value from `1` through `31536000` to enable it, then send
 `X-Dynamo-Session-ID` to keep related requests on one worker. Supplying the header
 without the TTL option provides session identity but does not enable router affinity.
 
-The first successfully dispatched request binds the session ID to its selected
-worker and, when available, data-parallel rank. Later requests exact-dispatch to
-that target without transport fallback. Concurrent requests can share a binding.
-Active requests prevent expiry. When a request lease ends after EOF, early drop,
-error, or cancellation, the idle timer restarts. A missing bound worker or a
-non-cancellation selection, setup, dispatch, or target-validation failure invalidates
-the binding.
-
-The configured value is the idle timeout. It is independent of
-`--router-ttl-secs` and `--router-predicted-ttl-secs`. Omit the session-affinity
-option to keep affinity disabled.
-
-If the bound worker disappears, Dynamo invalidates the binding so a subsequent
-selection can bind an available worker. Router restart clears all bindings. Bindings
-are not shared between frontend replicas.
-
-Direct mode still requires the phase-appropriate explicit worker ID on every
-affinity request. The stored binding validates that target but does not supply a
-missing ID. In disaggregated serving, prefill and decode use separate phase-local
-bindings. If no prefill router is active, only the decode or aggregated binding is
-created.
-
-Session affinity does not create a backend session or send lifecycle RPCs. There is
-no explicit unbind; idle expiry removes only router-local state. The same session
-ID is available to tracing and other explicitly configured consumers.
+The first affinity request creates one immutable binding from the session ID to a
+worker and, when available, a data-parallel rank. The binding is scoped to the
+existing endpoint and phase, so disaggregated prefill and decode routes remain
+separate. Later requests exact-dispatch to that target without transport fallback.
+An existing local or shared binding takes precedence over explicit routing headers;
+those headers are proposals only while the claim is absent. Direct mode therefore
+requires an explicit target for a new binding, but an existing binding supplies the
+target for later requests. Query-only requests remain read-only and do not create or
+close claims.
+
+With etcd or FileStore on a filesystem shared by all replicas, frontends coordinate
+through an immutable distributed claim. The existing-session hot path reads only the
+process-local cache. A cache miss reads shared storage first and attempts an atomic
+insertion only when the claim is absent. Racing frontends all cache and dispatch to
+the stored winner. Storage errors fail the request before scheduler bookkeeping or
+dispatch. MemoryStore coordinates only callers sharing the same process and store.
+Kubernetes discovery does not provide cross-process affinity and keeps process-local
+behavior.
+
+For distributed backends, `--router-session-affinity-ttl-secs` controls only
+process-local cache eviction. A cache miss after local eviction reloads the immutable
+claim. The claim itself follows the creating frontend's existing etcd lease or
+FileStore ownership lifetime; it is not a global idle-session timeout. Delete events
+eventually invalidate other frontend caches. Watch lag, disconnect, or restart clears
+the entire local affinity cache, and later requests reload claims on demand.
+
+`X-Dynamo-Session-Final: true` marks a terminal request. Dynamo routes that request
+normally, then evicts the closing frontend's cache entry and idempotently deletes the
+shared claim. Other replicas observe the delete eventually. Close must not race active
+requests, and callers must not use that session ID again. The same no-reuse rule
+applies after claim expiry. If the bound worker disappears while the claim exists,
+exact dispatch fails; start a new session with a new session ID.
+
+Global idle-session TTL, rebinding, dead-worker replacement, compare-and-swap updates,
+fencing, generations, broader `WorkerSet` affinity, and backend-tokenized path
+expansion are outside this contract. The setting remains independent of
+`--router-ttl-secs` and `--router-predicted-ttl-secs`; omit it to disable affinity.
 
 ### AIC Prefill Load Model
 

lib/llm/src/kv_router/prefill_router/admission.rs

@@ -20,7 +20,7 @@ use crate::{
         llm_backend::{LLMEngineOutput, PreprocessedRequest},
         timing::RequestTracker,
     },
-    session_affinity::SessionAffinityPushRouter,
+    session_affinity::{AffinityTarget, SessionAffinityPushRouter},
 };
 
 pub(super) enum InnerPrefillRouter {
@@ -35,7 +35,7 @@ impl InnerPrefillRouter {
         prepare: F,
     ) -> Result<(M, ManyOut<Annotated<LLMEngineOutput>>)>
     where
-        F: FnOnce(&mut PreprocessedRequest, u64, Option<u32>) -> Result<M>,
+        F: FnOnce(&mut PreprocessedRequest, AffinityTarget) -> Result<M>,
     {
         match self {
             InnerPrefillRouter::KvRouter(router) => {

lib/llm/src/kv_router/prefill_router/mod.rs

@@ -28,6 +28,7 @@ use crate::{
         preprocessor::{BootstrapInfo, PrefillResult, TraceLink},
         timing::{RequestPhase, RequestTracker},
     },
+    session_affinity::AffinityTarget,
 };
 
 mod activation;
@@ -224,8 +225,8 @@ impl
             .ok_or_else(|| anyhow::anyhow!(PrefillError::NotActivated))?;
         let prefill_result: Result<(PrefillOutcome, Option<RoutingConstraints>)> = async {
             let (prepared, prefill_stream) = router
-                .select_and_dispatch_prefill(prefill_context, |request, worker_id, dp_rank| {
-                    self.prepare_prefill_dispatch(request, worker_id, dp_rank)
+                .select_and_dispatch_prefill(prefill_context, |request, target| {
+                    self.prepare_prefill_dispatch(request, target)
                 })
                 .await?;
             let topology_constraints = prepared.topology_constraints;
@@ -327,9 +328,9 @@ impl PrefillRouter {
     fn prepare_prefill_dispatch(
         &self,
         request: &mut PreprocessedRequest,
-        worker_id: u64,
-        dp_rank: Option<u32>,
+        target: AffinityTarget,
     ) -> anyhow::Result<PreparedPrefill> {
+        let AffinityTarget { worker_id, dp_rank } = target;
         let endpoint_id = self.endpoint_id.get();
         let topology_constraints =
             self.preflight_kv_transfer_constraints(endpoint_id, worker_id)?;