Skip to content

feat(kv-router): one session affinity config and validator, mode on every host - #87

Draft
ishandhanani wants to merge 1 commit into
idhanani/session-affinity-unification-p5from
idhanani/session-affinity-unification-p6
Draft

ishandhanani wants to merge 1 commit into
idhanani/session-affinity-unification-p5from
idhanani/session-affinity-unification-p6

Conversation

@ishandhanani

@ishandhanani ishandhanani commented Oct 6, 2026 •

Copy link
Copy Markdown
Owner

Summary

Last PR of the session-affinity unification stack (base: idhanani/session-affinity-unification-p5). One configuration and one validator for every host, the session-affinity mode on the two hosts that lacked it, and the docs.

Config and validator

SessionAffinityConfig (TTL, mode, entry limit, session-id limit) is the one configuration. It gains the one validator: validate checks the TTL range and the limits before a table starts, and ttl_from_secs_f64 is the one range check for a TTL given in seconds (fractional allowed, non-finite rejected). The copies it replaces:

Before Now
SessionAffinity::validate_ttl (affinity.rs:209) SessionAffinityConfig::validate, run by with_config
SelectionServiceBuilder::build range check (service.rs:152) config.validate()
bindings session_affinity_ttl_from_secs (kv.rs:637-650) wraps ttl_from_secs_f64, naming the Python parameter
frontend_args.py hardcoded range (frontend_args.py:144) calls the new dynamo._core.validate_session_affinity_ttl_secs(ttl, "--router-session-affinity-ttl-secs")
EPP #[validate(range(...))] attribute SessionAffinityConfig::ttl_from_secs_f64 in Selector::new (invalid session affinity TTL context kept)

Error messages seen by users are unchanged where tests pin them (session_affinity_ttl_secs must be between 1 and 31536000 on the bindings, --router-session-affinity-ttl-secs must be between 1 and 31536000 on the frontend, session affinity TTL on the EPP).

Mode setting

hard or soft, hard by default (unchanged behavior when unset):

  • Standalone selection service: --session-affinity-mode (python -m dynamo.select_service), SelectionService(session_affinity_mode=...), SelectionServiceBuilder::session_affinity_mode, SelectionServiceConfig::session_affinity_mode.
  • EPP: DYN_EPP_SESSION_AFFINITY_MODE.
  • Frontend: --router-session-affinity-mode already existed; its reference page gains the missing ParamField.

Docs

  • Configuration and tuning: the mode table describes the pin rule (hard limits selection while eligible, re-binds when the worker leaves, rejected before dispatch when the request's own constraints exclude a live bound worker; soft is a preference), explicit targets, the full-table fallback, the shared session layer, and the v2 replication subject with its one-release dual-publish window.
  • Standalone selection: --session-affinity-mode, the v2 topic, hard/soft semantics for session_id.
  • Custom worker selection: hard affinity is a narrowed candidate set the policy still runs over, not an exact-target path that bypasses it.
  • Kubernetes KV routing on-ramp: DYN_EPP_SESSION_AFFINITY_MODE.
  • Frontend configuration reference: --router-session-affinity-mode.
  • lib/kv-router/src/services/selection/affinity/README.md: configuration section.

Behavior changes

None by default. The EPP's TTL error now surfaces when the selector starts rather than when the config is parsed (both at startup).

Validation

Run with RUSTC_WRAPPER= and a per-worktree CARGO_TARGET_DIR.

Command Result
cargo test -p dynamo-kv-router --features standalone-selection --lib 1291 passed, 2 ignored (+1 validator test)
cargo test -p dynamo-ext-proc 161 passed
cargo test -p dynamo-llm --lib kv_router 412 passed
cargo test -p dynamo-llm --lib session_affinity 50 passed
cargo test -p dynamo-custom-policy-builtin -p dynamo-mocker passed
cargo clippy (kv-router with standalone-selection,bench; llm, ext-proc, builtin, mocker) --all-targets -- -D warnings clean
cargo clippy --manifest-path lib/bindings/python/Cargo.toml --features select-service -- -D warnings clean
cargo fmt --all -- --check clean
python3 -m py_compile components/src/dynamo/frontend/frontend_args.py; validate_session_affinity_ttl_secs(x, "--router-session-affinity-ttl-secs") for 0, 0.5, NaN, 31536001 ok; each raises --router-session-affinity-ttl-secs must be between 1 and 31536000
maturin develop --uv --features select-service in <worktree>/.venv, then pytest lib/bindings/python/tests/test_kv_bindings.py -k "affinity or session" 13 passed, 20 deselected (includes the dynamo.select_service CLI rejection cases)
pytest lib/bindings/python/tests/test_kv_bindings.py (whole file) 33 passed
pytest tests/router/test_router_e2e_with_mockers.py -k session_affinity (etcd v3.5.21, nats-server v2.10.26, nats-py in the venv) 3 passed: test_mocker_session_affinity[etcd], [file], test_disagg_per_role_session_affinity
pytest tests/router/test_router_e2e_with_mockers.py (whole file, 56 tests) 50 passed, 6 skipped (the suite's own Flaky on CI: TCP round-robin skip marker). First pass with a select-service-only build failed 5 tests on missing build features (dynamo.indexer needs kv-indexer; the AIS perf model needs ais-forward-pass); rebuilt with select-service,kv-indexer,ais-forward-pass (a subset of the shipped wheel's feature set) and those 5 passed in 90 s

Related Issues

@ishandhanani
ishandhanani had a problem deploying to external_collaborator October 6, 2026 09:28 — with GitHub Actions Failure
@github-actions github-actions Bot added feat documentation Improvements or additions to documentation frontend labels Oct 6, 2026
@ishandhanani
ishandhanani added this pull request to stack #89 October 6, 2026 20:45
…very host

`SessionAffinityConfig` is the one session-affinity configuration and gains
the one validator: `validate` checks the TTL range and the limits before a
table starts, and `ttl_from_secs_f64` is the one range check for a TTL given
in seconds. The four copies of the TTL check (the table, the selection
service builder, the Python bindings, and the frontend's argument parser)
and the EPP's separate range attribute all go through it. The bindings
export `validate_session_affinity_ttl_secs` so `dynamo.frontend`'s argument
parser reports its own flag with the router's check instead of a copied
constant.

The session-affinity mode (`hard` or `soft`, `hard` by default) is now
settable on the standalone selection service (`--session-affinity-mode`,
`SelectionService(session_affinity_mode=...)`) and the EPP
(`DYN_EPP_SESSION_AFFINITY_MODE`), which were always `hard`.

Docs: the frontend's mode table describes the pin rule (a hard binding
limits selection while eligible, re-binds when its worker leaves, and is
rejected before dispatch when the request's own constraints exclude it; a
full table routes without affinity), the standalone service and EPP
reference pages gain the mode setting, the custom-policy guide describes
hard affinity as a narrowed candidate set rather than an exact-target path,
and the replication subjects and topics are documented with their
one-release dual-publish window.

Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>

This branch had an error being deployed

1 failed deployment
external_collaborator — be3de217 Deployed Oct 6, 2026 by ishandhanani via ok-to-test #272
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation feat frontend

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant