refactor(router): harden process-local session placement lifecycle

[qianlihuang]

Overview

Refactors the existing process-local worker session-affinity state behind a crate-private, target-agnostic placement lifecycle coordinator.

This remains a draft foundation for #11403, which is still dep:draft. It is not the Phase 1 multi-DC Global Router policy: it adds no proxy, public placement API, cross-replica claim, target catalog, base policy, or forwarding path. It preserves the process-local boundary restored by #11217 rather than reintroducing the distributed claims reverted from #11079.

Details

• Splits initialization into Reserved and Dispatching, fixes one provisional versioned target before forwarding, and fences completion with an attempt ID and target generation.
• Consumes explicit Accepted, DefinitelyNotAccepted, or Ambiguous outcomes. Only a definitely-not-accepted attempt rolls back for retry; a timeout or dropped in-flight dispatch guard is quarantined and never silently replayed.
• Allows a late accepted result or an explicit fenced resolution to settle an ambiguous attempt.
• Preserves committed-placement reuse, active-request leases, idle TTL, invalidation, bounded key/table limits, and opportunistic expired-entry reclamation.
• Stores targets behind Arc, removing the T: Clone requirement and avoiding caller-owned target clone/drop work while DashMap shards are locked.
• Tracks and restarts the cleanup task, uses an atomic hot-path health check, and runs table scans on the blocking pool.
• Keeps the coordinator and cancellation surface crate-private.
• Keeps the current worker-affinity behavior through an unversioned commit_already_accepted compatibility bridge after a response stream has been selected. That bridge is intentionally not the strict pre-dispatch contract for the future Global Router.

Intentional integration boundaries

• The primitive returns Capacity; the future Global Router adapter must implement DEP: Session-Aware Placement Policy for the Multi-DC Global Router #11403's stateless fallback. Existing worker affinity continues to report its current resource-exhausted behavior.
• Ambiguous entries remain quarantined until the forwarding owner resolves their outcome; this PR does not invent a retry or eviction policy for possibly accepted requests.
• Target generation is carried and fenced, but catalog freshness, health, eligibility, execution compatibility, and generation invalidation remain caller responsibilities.
• Wiring begin_dispatch across the production network await is deferred until the forwarding layer can classify accepted, definitely-not-accepted, and ambiguous transport/stream outcomes and own recovery. Current worker affinity remains behavior-compatible.
• Trusted composite key derivation, routing-domain discovery, base-policy selection, request forwarding/status mapping, requests without session identity, placement metrics, parent preference, spillover/movement, multi-replica state, restart recovery, and Relay/CKF integration are deferred to the Global Router work in DEP: Session-Aware Placement Policy for the Multi-DC Global Router #11403/DEP: Multi-DC KV-Aware Request Routing #11225.

Upstream review

Reviewed canonical main at 0ee5aa6f9a156609863eef62f5decb241f64287c. The only upstream change since this PR's base is #11569 in lib/llm/src/http/service/metrics.rs, which does not overlap this change. #11403 and #11225 remain dep:draft; #11217 remains the process-local boundary, and #11435 remains router-local CKF work rather than Global Router transport/recovery plumbing.

Where should the reviewer start?

Start with lib/llm/src/session_placement/coordinator.rs, especially the dispatch-outcome and ambiguous-state transitions. Then review lib/llm/src/session_affinity/coordinator.rs for the behavior-preserving adapter and lib/llm/src/session_placement/tests.rs for concurrency and fencing coverage.

Validation

• cargo fmt --all -- --check
• cargo clippy --locked -p dynamo-llm --no-default-features --lib -- -D warnings
• cargo test --locked -p dynamo-llm --no-default-features --lib session_ — 60 passed
• cargo test --locked -p dynamo-llm --no-default-features --lib push_router — 19 passed
• pre-commit run --files <changed files>

Related Issues

• Relates to DEP: Session-Aware Placement Policy for the Multi-DC Global Router #11403
• Context: DEP: Multi-DC KV-Aware Request Routing #11225, revert(router): keep session affinity local [DYN-3249] #11217, feat(router): coordinate session affinity across replicas [DYN-3249] #11079, feat(kv-router): add relay-shaped transposed CKF indexer [DYN-3584] #11435

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[github-actions]

👋 Hi qianlihuang! Thank you for contributing to ai-dynamo/dynamo.

Just a reminder: The NVIDIA Test Github Validation CI runs an essential subset of the testing framework to quickly catch errors.Your PR reviewers may elect to test the changes comprehensively before approving your changes.

🚀

[github-actions]

This PR is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.