Skip to content

[sgl-router] policies_reorg: affinity policy and the resolver affinity group - #40274

Closed
sherlockwu wants to merge 1 commit into
router/policies-reorg-3-factoryfrom
router/policies-reorg-4-affinity
Closed

sherlockwu wants to merge 1 commit into
router/policies-reorg-3-factoryfrom
router/policies-reorg-4-affinity

Conversation

@sherlockwu

@sherlockwu sherlockwu commented Sep 19, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

Fourth PR of the engine-selection redesign, on top of #40273. Adds session and routing-key affinity on the new path, including the design's affinity-first group and global session modes.

Modifications

  • policies_reorg/affinity.rs: AffinityPolicy { kind: Session | Sticky, admission, store, global, fallback }. Session and sticky differ only in which request key they read and which fallback they get, so they share one implementation. An existing binding among the candidates is reused when it passes admission. Otherwise the fallback chooses among admitted candidates and the choice is bound (AffinityStore::bind keeps a concurrent binding that is still a candidate). In HitRequired mode the policy never falls back or binds, and it reports a binding held by a non-candidate as a rejection so the resolver can preserve it.
  • PickRequest gains mode, affinity_enabled, and affinity_key(kind, global, value); keys are stage/kind/scope/value with scope global or the bucket id.
  • buckets_reorg.rs: Pool.affinity: Option<Arc<dyn Policy>> adds the affinity group ahead of the size buckets, over the stage pool filtered by each engine's own bucket context limit and SLO rule under slo_first. Pool.preserve_global_binding implements global-preserve: after the affinity group rejects a stranded binding, later groups run with affinity disabled. Bucket mode has no affinity group and scopes keys per bucket; global-rebind has the group and lets a size bucket rebind.
  • admission::Admission::delegate runs a nested policy over the admitted candidates at the owner's placement; nested fallbacks use AllowAll.
  • factory.rs: Dependencies { affinity: Arc<AffinityStore> }. Session-aware falls back to power-of-two; sticky to its configured fallback kind. The affinity group is enabled for sticky and for session-aware in a global mode when buckets are configured.
  • The trait's unused pick_fallback default is gone; fallback() remains the introspection hook.

Metrics for sticky and session outcomes are deferred to the switch PR, where the legacy counters are reconciled.

Checklist

  • cargo clippy --locked --workspace --all-targets -- -D warnings clean
  • cargo test --lib policies_reorg (8) and cargo test --test component policies_reorg (7) pass

🤖 Generated with Claude Code


CI States

Latest PR Test (Base): ✅ Run #35410405514
Latest PR Test (Extra): ❌ Run #35410405349
Latest PR Test (AMD ROCm 10): ➖ No AMD PR run found for this commit.

…y group

One AffinityPolicy serves session-aware and sticky routing: reuse an admitted
binding for the request's key, otherwise let the fallback choose and bind its
pick. HitRequired returns only an existing hit. The resolver gains an
optional affinity group ahead of the size buckets, probed in HitRequired
mode over the whole stage pool, and global-preserve stops later buckets from
binding once a stranded binding is found. The factory wires both from the
session and sticky settings.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@sherlockwu

Copy link
Copy Markdown
Collaborator Author

Closing: the series stops at the power-of-two PR (#40271) for now.

@sherlockwu sherlockwu closed this Sep 19, 2026
@sherlockwu
sherlockwu deleted the router/policies-reorg-4-affinity branch September 19, 2026 01:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant