docs(observability): list the Global Frontend affinity metrics

Cargo.toml

@@ -18,6 +18,7 @@ members = [
     "lib/global-discovery-service",
     "lib/global-frontend",
     "lib/global-router",
+    "lib/global-router-store-dynamodb",
     "lib/global-view-broadcaster",
     "lib/global-view-receiver",
     "lib/router-plugins/catalog",
@@ -82,6 +83,7 @@ dynamo-data-gen = { path = "lib/data-gen", version = "1.7.0" }
 dynamo-kv-hashing = { path = "lib/kv-hashing", version = "1.7.0" }
 dynamo-global-common = { path = "lib/global-common", version = "1.7.0" }
 dynamo-global-router = { path = "lib/global-router", version = "1.7.0" }
+dynamo-global-router-store-dynamodb = { path = "lib/global-router-store-dynamodb", version = "1.7.0" }
 dynamo-http-server = { path = "lib/http-server", version = "1.7.0" }
 dynamo-memory = { path = "lib/memory", version = "1.7.0" }
 dynamo-mocker = { path = "lib/mocker", version = "1.7.0" }

docs/fern/pages/reference/observability/metrics-catalog.mdx

@@ -954,6 +954,28 @@ Emitted by the Global Frontend at `/metrics` on its serving port. It forwards ea
   **Experimental.** Requests this Global Frontend has sent to each pool and not yet finished. Other replicas count their own. Labeled by `pool` and `region`; read at scrape time, so a pool removed from the configuration loses its series.
 </ParamField>
 
+The following are served only with `--router-policy affinity`. They cover session requests, the ones that carry `x-dynamo-session-id`.
+
+<ParamField path="dynamo_global_frontend_overhead_affinity_store_ms" type="histogram">
+  **Experimental.** Time each session request spent in affinity storage, in milliseconds: the lookup, any write and its retry, up to `--affinity-timeout-ms`. A request whose budget ran out records the budget. Requests that skip storage while the breaker is open are not recorded. Named and bucketed like [`dynamo_router_overhead_shared_cache_query_ms`](#per-request-routing-overhead).
+</ParamField>
+
+<ParamField path="dynamo_global_frontend_affinity_lookups_total" type="counter">
+  **Experimental.** Session requests whose affinity lookup found a live pin among the candidate pools (`outcome="hit"`) or not (`miss`).
+</ParamField>
+
+<ParamField path="dynamo_global_frontend_affinity_store_errors_total" type="counter">
+  **Experimental.** Session requests that routed on headroom alone because affinity was lost, one count per request. Labeled by `kind`: `backend` (the store returned an error), `timeout` (the budget ran out), `conflict` (conditional writes kept losing), or `breaker_open` (storage was not consulted, after repeated failures).
+</ParamField>
+
+<ParamField path="dynamo_global_frontend_affinity_store_writes_total" type="counter">
+  **Experimental.** Conditional writes sent to the affinity store. Labeled by `reason`: `create` (a session without a live pin), `move` (a session moving pools), `refresh` (a kept pin past half its idle window) or `repair` (a record no reader could decode). A request that keeps a fresh pin writes nothing.
+</ParamField>
+
+<ParamField path="dynamo_global_frontend_affinity_cache_total" type="counter">
+  **Experimental.** Affinity store reads by whether the in-process cache answered them (`outcome="hit"`) or the store did (`miss`), and cache entries dropped (`invalidation`). Only a DynamoDB store is cached.
+</ParamField>
+
 ## Mocker Metrics
 
 The mocker (`python -m dynamo.mocker`) simulates an inference engine. It emits imitation engine metrics with simulated values on `DYN_SYSTEM_PORT`, choosing the set by `--engine-type`. They carry the engine's usual labels plus `dynamo_namespace`, `dynamo_component`, `dynamo_endpoint`, `worker_id`, `model`, and `model_name`. These imitate engine metrics, so this table has no Labels, Description, or Status column; each metric means what it means in the engine it imitates.

lib/global-frontend/Cargo.toml

@@ -20,17 +20,21 @@ dynamo-global-common = { workspace = true, features = ["grpc"] }
 dynamo-global-view-receiver = { path = "../global-view-receiver" }
 dynamo-global-discovery-client = { workspace = true }
 dynamo-global-router = { workspace = true }
+dynamo-global-router-store-dynamodb = { workspace = true }
 # For the PoolRelay protocol and the session header.
 dynamo-llm = { workspace = true }
 dynamo-runtime = { workspace = true }
 
 anyhow = { workspace = true }
+async-trait = { workspace = true }
+aws-config = { workspace = true }
+aws-sdk-dynamodb = { workspace = true }
 parking_lot = { workspace = true }
 prost = { workspace = true }
 thiserror = { workspace = true }
 tokio-stream = { workspace = true, features = ["net"] }
 axum = { workspace = true }
-clap = { version = "4.5.49", features = ["derive"] }
+clap = { version = "4.5.49", features = ["derive", "env"] }
 futures = { workspace = true }
 hyper-util = { workspace = true, features = ["server-auto", "service", "tokio"] }
 prometheus = { workspace = true }

lib/global-frontend/README.md

@@ -54,8 +54,10 @@ session or a request without a session ID, to the least full pool in the
 nearest region. It goes elsewhere only when another pool has more than
 `--headroom-factor` (default 2) times that pool's spare capacity, choosing the
 nearest such pool, or when that pool is full, choosing the nearest pool with any
-room. The session's pin moves with it. Pins are per process and expire after
-`--session-idle-secs` (default 1800) without a request.
+room. The session's pin moves with it. Pins expire after
+`--session-idle-secs` (default 1800) without a request, and are held in this
+process unless `--affinity-store dynamodb` shares them (see
+[Session affinity storage](#session-affinity-storage)).
 
 `affinity` needs `--region`, matching the pool's region (the cluster name for
 explicit pools). It reads each pool's load from the pool's PoolRelay, given per pool
@@ -87,6 +89,125 @@ matching window arrives. The round trip to each region is the median TCP connect
 its pools, measured every 30 seconds; the request's own region counts as
 zero.
 
+### Session affinity storage
+
+`--affinity-store` decides where `affinity`'s session pins live. `memory`, the
+default, keeps them in this process: a restart or a second replica starts with
+none. `dynamodb` keeps them in a DynamoDB table, so replicas of this binary
+agree on a session's pool and a restart resumes where it left off. No other
+component reads or writes the table, and nothing else in Dynamo depends on it.
+
+Affinity is advisory throughout. A lookup or write that fails, or that runs
+past `--affinity-timeout-ms` (default 20), leaves the request on the pool the
+headroom rules already chose: no request is refused because storage is down.
+After `--affinity-breaker-failures` consecutive failures (default 5) the
+frontend stops consulting storage for `--affinity-breaker-cooldown-ms` (default
+5000), so an outage costs one deadline per cooldown rather than one per
+request; a single request then retests it.
+
+A request that keeps its session's pool writes nothing. The binding is written
+when a session is new or its pin has expired, when it moves pools, when an
+undecodable row is repaired, and once the pin is past half its idle window
+(`--session-idle-secs`), which keeps an active session from expiring. So most
+turns of a conversation only read, and parallel requests of one session, such
+as concurrent tool calls, do not conflict with each other.
+
+Reads are cached in this process for `--affinity-cache-ttl-ms` (default 1000,
+0 to disable) across at most `--affinity-cache-entries` sessions (default
+100000), so a busy session reads the table about once per TTL rather than once
+per request. When another replica moves a session, this replica may keep
+routing it to the old pool until its cached entry ages out, which is at most
+the TTL. A stale entry never overwrites the newer binding: a write made from it
+loses its conditional write and is re-read.
+
+#### Table
+
+Provision the table yourself; the frontend never creates it. It needs a string
+partition key `itemHashKey`, a string sort key `sortKey`, and TTL enabled on
+`expiresAt`. On-demand capacity suits the access pattern: one `GetItem` per
+cache miss, and a `PutItem` only when a binding changes or needs a refresh.
+
+A Global Table (MREC) uses exactly this schema, and its pins are soft: last
+writer wins between regions. Conditional writes serialize against the
+contacted regional replica only, so when one session reaches frontends in two
+regions before either's write has replicated, both writes apply and
+replication keeps the later one. Since a kept pin is not rewritten, the
+session then settles on that pool in both regions; it is not a global lock.
+
+```bash
+aws dynamodb create-table --table-name dynamo-global-affinity \
+  --billing-mode PAY_PER_REQUEST \
+  --attribute-definitions \
+    AttributeName=itemHashKey,AttributeType=S \
+    AttributeName=sortKey,AttributeType=S \
+  --key-schema \
+    AttributeName=itemHashKey,KeyType=HASH \
+    AttributeName=sortKey,KeyType=RANGE
+aws dynamodb update-time-to-live --table-name dynamo-global-affinity \
+  --time-to-live-specification Enabled=true,AttributeName=expiresAt
+```
+
+Rows hold `poolId`, `revision`, `updatedAt`, `expiresAt` and `schemaVersion`.
+Keys are BLAKE3 digests, so no session ID, credential or model name is stored
+in the clear. A row that no reader can decode — a newer `schemaVersion`, or a
+damaged write — is replaced by the next request for that session rather than
+stranding it, which is also what lets a mixed-version rollout recover on its
+own. TTL deletion is DynamoDB's own, within its usual delay; a row past
+`expiresAt` is treated as expired on read whether or not it has been swept.
+
+#### Scope and tenancy
+
+`--affinity-scope` is the namespace bindings are shared in, and every replica
+that should agree must pass the same one. It is deployment configuration and is
+never derived from a request. `--affinity-policy` (default `headroom-v1`)
+separates incompatible placement policies within a scope, so rolling out a new
+policy does not inherit the old one's pins.
+
+Within a scope, a session's binding is also keyed by a digest of the request's
+`authorization` header, so two tenants that reuse one session ID never share a
+binding; requests with no credential share one `anonymous` namespace. Note what
+this is not: the Global Frontend does not validate credentials, and selection
+happens before the pool authorizes the request, so a request the pool will
+reject can still create or refresh a binding. Treat the table as writable by
+anyone who can reach the frontend, and keep the frontend behind whatever admits
+traffic to the deployment. Give each tenant its own `--affinity-scope` where
+their bindings must not share a table at all.
+
+#### IAM
+
+The frontend's role needs only item access on the table:
+
+```json
+{
+  "Effect": "Allow",
+  "Action": ["dynamodb:GetItem", "dynamodb:PutItem"],
+  "Resource": "arn:aws:dynamodb:<region>:<account>:table/dynamo-global-affinity"
+}
+```
+
+Credentials and region come from the standard AWS provider chain, with
+`--affinity-ddb-region` overriding the region and `--affinity-ddb-endpoint`
+pointing at DynamoDB Local for development. `--affinity-consistent-reads`
+(default true) asks for strongly consistent reads within the contacted region.
+
+#### Starting it
+
+```bash
+cargo run -p dynamo-global-frontend -- --port 8000 \
+  --router-policy affinity --region ohio \
+  --pool ohio/dynamo/blue=Qwen/Qwen3-0.6B@http://127.0.0.1:8001 \
+  --pool oregon/dynamo/green=Qwen/Qwen3-0.6B@http://127.0.0.1:8002 \
+  --pool-relay ohio/dynamo/blue=http://127.0.0.1:50051 \
+  --pool-relay oregon/dynamo/green=http://127.0.0.1:50052 \
+  --affinity-store dynamodb \
+  --affinity-scope production \
+  --affinity-ddb-table dynamo-global-affinity
+```
+
+Every flag has an environment variable (`DYN_GLOBAL_AFFINITY_STORE`,
+`DYN_GLOBAL_AFFINITY_SCOPE`, `DYN_GLOBAL_AFFINITY_DDB_TABLE` and the rest), for
+deployments that configure the frontend through the environment.
+
 ## Relay-initiated state
 
 Choose the connection direction for the deployment. Router-initiated mode uses
@@ -132,7 +253,20 @@ named `dynamo_global_frontend_*`: requests per pool by response status,
 `no_response` when the pool sends no response, or `cancelled` when the client
 leaves before the pool's response headers arrive; requests answered without a
 pool; time to first byte and duration per pool; and requests in flight per
-pool. The `pool` label is the pool's
+pool. With `--router-policy affinity` it also serves:
+
+- `overhead_affinity_store_ms`, the time each session request spent in affinity
+  storage, named and bucketed like the local router's
+  `dynamo_router_overhead_shared_cache_query_ms`;
+- `affinity_lookups_total` by `outcome`: `hit` or `miss`;
+- `affinity_store_errors_total` by `kind`, counting each session request that
+  routed without affinity: `backend`, `timeout`, `conflict`, or `breaker_open`
+  (storage was not consulted);
+- `affinity_store_writes_total` by `reason`: `create`, `move`, `refresh` or
+  `repair`;
+- `affinity_cache_total` by whether the in-process cache answered a read.
+
+The `pool` label is the pool's
 `<cluster>/<namespace>/<DGD name>` and `region` its region. On requests, `local`
 says whether the pool is in `--region` (`unknown` without it), which gives the
 share of requests kept in the region. Nothing is labelled from the request