Repository navigation
feat(kv-router): DisaggCoordinator with linked-booking compensation - #20
Closed
ishandhanani wants to merge 1 commit into
Closed
ishandhanani wants to merge 1 commit into
ishandhanani wants to merge 1 commit into
Conversation
ishandhanani
had a problem deploying
to
external_collaborator
September 3, 2026 08:42 — with
GitHub Actions
Failure
Phase 3 step 1 of the runtime-optional frontend roadmap: the contract-5 disaggregated coordination as a runtime-free component over two selection pools, shared by every host (frontend, EPP, standalone service). DisaggCoordinator books a request decode-first: advisory decode preview (potential_decode_blocks, decode_busy, cached tokens), optional advisory prefill busy read, conditional-disaggregation policy plus the decode busy gate, decode commit pinned to the previewed worker, then prefill commit constrained to the decode worker's KV-transfer domain (its kv_transfer_domain and topology_domains become a required or preferred dynamo.topology taint per kv_transfer_enforcement). Cross-pool atomicity is explicitly waived: the two ledgers are independent and every partial state is compensated and recorded in the plan's LinkedBookingState transitions. Prefill failure after decode is booked follows PrefillFailurePolicy (free decode and fail, or bypass on the decode worker); prefill_complete frees prefill early; release frees the rest. SelectionPool is the seam: SelectionService implements it; the EPP can implement it over its own client. Not yet wired into PrefillRouter (the production path still books prefill first); that switch is gated on the replay-parity disaggregated lifecycle campaign. Tests: linked path with exact transition sequence and per-pool load checks, policy bypass, decode-gate veto, prefill failure under both compensation policies, decode failure booking nothing, KV-transfer-domain constraint steering prefill, and incomplete KV-transfer metadata failing closed. Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
ishandhanani
force-pushed
the
idhanani/p3-disagg-coordinator
branch
from
September 3, 2026 09:05
68fefc5 to
dc94857
Compare
ishandhanani
had a problem deploying
to
external_collaborator
September 3, 2026 09:05 — with
GitHub Actions
Failure
ishandhanani
had a problem deploying
to
external_collaborator
September 3, 2026 09:05 — with
GitHub Actions
Failure
This was referenced Sep 3, 2026
Owner
Author
This branch had an error being deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 3, step 1 (contract 5) of the runtime-optional frontend roadmap. Stacked on #19.
Summary
lib/kv-router/src/services/selection/coordinator.rs: the linked-booking state machine for disaggregated serving over two selection pools, implemented once (runtime-free) so the frontend, the EPP, and the standalone service share it. This is the roadmap's highest-severity risk item, so it lands first as a self-contained, exhaustively tested component before the productionPrefillRouteris switched onto it.select(uses feat(kv-router): advisory select with worker load and busy evaluation #12'spotential_decode_blocks/decode_busy/ cached tokens) -> optional advisory prefill busy read (worker_load.prefill_busy) only when the policy consumes it ->ConditionalDisaggPolicy(conditional_disagg.rs) + the decode busy gate (same rule asconditional_bypass.rs::decode_gate_allows_bypass) -> decodeselect_and_reservepinned to the previewed worker -> prefillselect_and_reservewithrouting_constraintsextended by the decode worker's KV-transfer domain (kv_transfer_domain+topology_domains->dynamo.topology/<domain>=<value>, required or preferred perkv_transfer_enforcement, same derivation asModelManager::get_kv_transfer_routing_constraints).LinkedBookingStatetransitions on the plan): decode failure books nothing; prefill failure after decode is booked followsPrefillFailurePolicy(FreeDecodedefault, orBypassOnDecode);prefill_completefrees prefill early;releasefrees whatever remains. Cross-pool atomicity is waived by design; decode-first greedy pairing accepted.SelectionPooltrait is the seam;SelectionServiceimplements it. Selection ids derive as<id>/decodeand<id>/prefill.build_decode_router_override(no prompt-side load, no KV-reuse assumption, zero overlap credit unless bypass).standalone-selection.md("Disaggregated coordination", with the compensation table).Not in this PR: switching
PrefillRouter/EPP onto the coordinator. The production path still books prefill first and merges topology constraints into decode; flipping that order changes booking behavior under NIXL handoff and is gated on the replay-parity disaggregated lifecycle campaign (Phase 3 exit gate).Validation
cargo test -p dynamo-kv-router --features standalone-selection --lib: 1077 passed. New (8):linked_plan_books_both_pools_and_releases_in_order(exact transition sequenceIdle -> DecodePreviewed -> DecodeCommitted -> Linked -> PrefillReleased -> Released, per-pool active-request counts at each step, idempotentprefill_complete),policy_bypass_books_decode_only,decode_busy_gate_vetoes_a_policy_bypass,prefill_failure_frees_decode_by_default,prefill_failure_can_bypass_on_the_decode_worker,decode_failure_books_nothing,prefill_is_constrained_to_the_decode_kv_transfer_domain(6 plans all land in the decode worker's zone),incomplete_kv_transfer_metadata_is_an_error_and_frees_decode.cargo clippy --features standalone-selection --all-targets: only the pre-existingregistry.rsdead-code warning.cargo fmtanddocs_lint.pyclean.