[router] set manual policy (sticky + min_load) as default agentic routing policy - #1690
Merged
Zhichenzzz merged 1 commit intoJul 20, 2026
Conversation
yueming-yuan
requested review from
Shi-Dong,
fzyzcjy and
guapisolo
as code owners
July 15, 2026 22:39
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request updates the routing logic in 'miles/rollout/sglang_rollout.py' to support the 'manual' router policy in addition to 'consistent_hashing' when using session IDs. The feedback suggests updating the outdated inline comments on lines 195 and 316 to accurately reflect this new behavior and avoid confusion.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| # Use session_id for consistent hashing routing if router uses consistent_hashing policy | ||
| headers = None | ||
| if args.sglang_router_policy == "consistent_hashing" and sample.session_id: | ||
| if args.sglang_router_policy in ("consistent_hashing", "manual") and sample.session_id: |
Contributor
There was a problem hiding this comment.
The comment on line 195 still specifically mentions the consistent_hashing policy. Since support for the manual policy has been added, please update the comment to reflect this change (e.g., by generalizing it to session-based routing).
|
|
||
| # Generate a unique session_id for each sample in the group (consistent hashing only) | ||
| if args.sglang_router_policy == "consistent_hashing": | ||
| if args.sglang_router_policy in ("consistent_hashing", "manual"): |
Contributor
There was a problem hiding this comment.
The comment on line 316 still states (consistent hashing only). Since the manual policy is now also supported, please update this comment to prevent misleading future readers.
yueming-yuan
requested review from
Zhichenzzz,
maocheng23 and
yushengsu-thu
as code owners
July 15, 2026 23:17
yueming-yuan
force-pushed
the
yueming/router-manual-policy
branch
from
July 15, 2026 23:21
8f45ecb to
bb5e2b3
Compare
yueming-yuan
changed the title
yueming-yuan
force-pushed
the
yueming/router-manual-policy
branch
from
July 16, 2026 23:36
bb5e2b3 to
ff40ec5
Compare
yueming-yuan
added a commit
that referenced
this pull request
Jul 17, 2026
…ollout stack - rename Sample.session_id to Sample.routing_key to disambiguate from the session server's and p2p transfer engine's session ids - unify all routing-key sites behind policy_uses_routing_key, covering the manual policy (#1690) as well: both stacks' group assignment and eval, single_turn/multi_turn/legacy generate headers, prefill recompute - drop the MILES_EXPERIMENTAL_ROLLOUT_REFACTOR=1 restriction on --sglang-router-policy Keyless-request enforcement lives router-side (radixark/sgl-router-for-miles#8), which covers every client including proxy layers that bypass miles' http utils.
Collaborator
Author
|
Heads up on two interactions with #1703 / radixark/sgl-router-for-miles#8:
|
yueming-yuan
force-pushed
the
yueming/router-manual-policy
branch
from
July 17, 2026 22:27
ff40ec5 to
dd3dc25
Compare
yueming-yuan
changed the base branch from
main
to
yueming/new-stack-consistent-hashing
July 17, 2026 22:27
yueming-yuan
added a commit
that referenced
this pull request
Jul 17, 2026
…ollout stack - rename Sample.session_id to Sample.routing_key to disambiguate from the session server's and p2p transfer engine's session ids - unify all routing-key sites behind policy_uses_routing_key, covering the manual policy (#1690) as well: both stacks' group assignment and eval, single_turn/multi_turn/legacy generate headers, prefill recompute - drop the MILES_EXPERIMENTAL_ROLLOUT_REFACTOR=1 restriction on --sglang-router-policy Keyless-request enforcement lives router-side (radixark/sgl-router-for-miles#8), which covers every client including proxy layers that bypass miles' http utils.
yueming-yuan
force-pushed
the
yueming/new-stack-consistent-hashing
branch
from
July 17, 2026 22:43
d1cf52b to
289ca4b
Compare
yueming-yuan
force-pushed
the
yueming/router-manual-policy
branch
from
July 17, 2026 22:43
dd3dc25 to
f85fcc6
Compare
yueming-yuan
added a commit
that referenced
this pull request
Jul 17, 2026
…ollout stack - rename Sample.session_id to Sample.routing_key to disambiguate from the session server's and p2p transfer engine's session ids - unify all routing-key sites behind policy_uses_routing_key, covering the manual policy (#1690) as well: both stacks' group assignment and eval, single_turn/multi_turn/legacy generate headers, prefill recompute - drop the MILES_EXPERIMENTAL_ROLLOUT_REFACTOR=1 restriction on --sglang-router-policy Keyless-request enforcement lives router-side (radixark/sgl-router-for-miles#8), which covers every client including proxy layers that bypass miles' http utils.
yueming-yuan
force-pushed
the
yueming/new-stack-consistent-hashing
branch
from
July 17, 2026 22:44
289ca4b to
dd2a6cc
Compare
yueming-yuan
force-pushed
the
yueming/router-manual-policy
branch
from
July 17, 2026 22:44
f85fcc6 to
6a388e9
Compare
yueming-yuan
changed the title
Zhichenzzz
merged commit Jul 20, 2026
a171974
into
yueming/new-stack-consistent-hashing
34 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
before:
after:
🤖 Generated with Claude Code