Repository navigation
feat(kv-router): router hints from the selection service - #16
Closed
ishandhanani wants to merge 1 commit into
Closed
ishandhanani wants to merge 1 commit into
ishandhanani wants to merge 1 commit into
Conversation
Port the frontend KvRouter router-hint chain into the selection service so a booked selection can tell the chosen worker which same-role peer holds a longer cached prefix and where to fetch it from. - Catalog records carry router_hint_worker_type and per-global-DP-rank router_hint_source_control_endpoints (WorkerRequest, WorkerPatchRequest, WorkerCatalogRecord, SelectionWorkerConfig), and SelectionWorkerConfig implements WorkerConfigLike::router_hint_metadata_for_dp_rank on them. The frontend derives the same two facts from the runtime_data capability flag, worker type key, and endpoints map. - Indexer::find_tiered_matches_with_options honors LowerTierQueryOptions::retain_router_hint_chain on a local event-driven primary and ignores it elsewhere (approximate writes and remote primaries cannot retain the matched chain). - SelectionCore retains the chain for bookings when the partition has a hint-capable schedulable worker, passes router_hint_candidates to the scheduler, and builds SelectResponse::router_hint with a port of router_hint_for_selection: a same-role source (worker or cache owner) that beats the target's own cached prefix and advertises a control endpoint. Query-only selections never carry a hint. Not ported: the frontend's exclusion of kv_event_source_mode=state_agent_v2 sources; the catalog record has no such field yet. Tests: two hint-capable workers, a booking on the cold worker receives a hint naming the warm worker's endpoint and both block hashes; a booking on the warm worker and a query-only select do not; workers without hint metadata never produce one. Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
ishandhanani
had a problem deploying
to
external_collaborator
September 3, 2026 07:03 — with 
GitHub Actions
Failure
GitHub Actions
Failure
github-actions
Bot
added
feat
documentation
This was referenced Sep 3, 2026
Closed
Owner
Author
This branch had an error being deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 0 (selection-core parity), item 2c: the router-hint chain (
core/mod.rs:772-773on main hardcodedrouter_hint_candidates: None, retain_router_hint_chain: false). Stacked on #15. This completes the Phase 0 list from the roadmap.Summary
router_hint_worker_type: Option<String>androuter_hint_source_control_endpoints: HashMap<u32, String>onWorkerRequest,WorkerPatchRequest,WorkerCatalogRecord, andSelectionWorkerConfig.SelectionWorkerConfigimplementsWorkerConfigLike::router_hint_metadata_for_dp_rankfrom them, which is whatModelRuntimeConfigderives fromruntime_data(router_hintcapability flag +router_hint_worker_type+router_hint_source_control_endpoints,lib/llm/src/local_model/runtime_config.rs:451). Presence of a non-empty worker type is the capability.Indexer::find_tiered_matches_with_options(sequence, LowerTierQueryOptions);find_tiered_matchesdelegates. Retention is honored only whensupports_router_hint_chain_retention()(local event-driven primary, no side indexer, not remote), same rule as the frontendIndexer::supports_router_hint_chain_retention.retain_router_hint_chain = book && indexer.supports_router_hint_chain_retention() && catalog.has_router_hint_capable_workers(key); candidates flow throughScheduleRequestand back viaSchedulingResponse.router_hint_candidates;router_hint_for_selectionis a port ofKvRouter::router_hint_for_selection(lib/llm/src/kv_router.rs:1022) overSelectionWorkerConfig, including the cache-owner branch throughRouterHintRootCandidates.routing_snapshot. Result lands onSelectResponse.router_hint: Option<RouterHint>(serdeRouterHintfromrouter_hint.rs).kv_event_source_mode == "state_agent_v2"; the catalog has no such field yet. Noted in the commit.router_hintresponse field instandalone-selection.md.Validation
cargo test -p dynamo-kv-router --features standalone-selection --lib: 1069 passed. New:booked_selection_attaches_router_hint_from_a_better_source(event-driven indexer, two hint-capable workers, worker 1 holds a 2-block prefix; a booking pinned to worker 2 getssource_control_endpoint == tcp://worker-1:9000and 2 block hashes; a booking on worker 1 and a query-only select get none),router_hint_needs_capable_workers.cargo clippy --features standalone-selection --all-targets: only the pre-existingregistry.rsdead-code warning.cargo fmtanddocs_lint.pyclean.serde(default)), so no bindings change.