feat(router): ship opt-in soft affinity load guard

[JasperChan929]

Summary

Ship an opt-in soft session-affinity worker-selection policy in the built-in catalog. When a parent-group binding has an eligible target, the policy keeps that worker until its active-request count exceeds max_active_requests and another eligible worker is lower by at least move_margin. Dynamo's existing affinity coordinator continues to own and update the binding; this PR adds no parent-key logic or router CLI flag.

For example, with max_active_requests: 7 and move_margin: 2, source/alternative loads of 7/0 and 8/7 retain the source, while 8/6 selects the alternative. The shipped defaults are 32/2. The policy is selected through existing router-policy YAML.

The existing soft-pin-repin example requires a custom catalog build and moves once the threshold is crossed even when the alternative is equally or more loaded. This proposal makes the guarded behavior available in a stock build. It is separate from #15139's binary retain/ignore soft-target switch and #14596's generic two-tier cache/load weights.

Validation

• cargo test --locked -p dynamo-custom-policy-builtin: 11 passed.
• cargo clippy --locked -p dynamo-custom-policy-builtin --all-targets -- -D warnings: passed.
• cargo fmt --all -- --check and git diff --check: passed.
• Fern docs lint: 0 errors; 8 warnings in untouched files.
• The three touched upstream files matched current main's blob hashes before publication.

Limits and next validation

This is a draft capability PR, not a measured speedup. Active-request count is a coarse load proxy; the policy does not know hotspot duration or cache residency. Unbound requests choose the least-loaded eligible worker without consulting cache overlap. Historical GPU measurements motivate the question but used an earlier router revision.

Before requesting merge, compare this policy on current main against hard affinity, two-tier with its existing tunable thresholds, and applicable soft-affinity configuration. Use equal capacity, two physical GPUs, balanced source/order, short and sustained hotspots, plus explicit stay/move controls. Report group-max TTFT, request latency, cached/computed tokens, placement, background impact, and failures. Keep unfavorable valid samples and stop if existing configuration already covers the workload or this policy regresses short hotspots.

Where should the reviewer start?

Review lib/router-plugins/builtin/src/soft_affinity_load_guard.rs for the threshold and margin decisions, then lib/router-plugins/builtin/src/lib.rs for registration and YAML resolution.

Related issues

Relates to #14536. Builds on merged #14521.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[github-actions]

👋 Hi JasperChan929! Thank you for contributing to ai-dynamo/dynamo.

Just a reminder: The NVIDIA Test Github Validation CI runs an essential subset of the testing framework to quickly catch errors.Your PR reviewers may elect to test the changes comprehensively before approving your changes.

🚀