[PeaBrane]

Summary

• Adds a runtime-free Dynamo selection service under kv-router with worker catalog reconciliation, selection/reservation APIs, readiness/load/scoring endpoints, and clear V1 retry semantics.
• Wires the Python dynamo.select_service entrypoint and extends router mocker tests to exercise standalone selector scoring.

Validation

• cargo test -p dynamo-kv-router --features standalone-selection services::selection --no-default-features
• cargo test --features select-service --lib run_select_service from lib/bindings/python

---

Summary by CodeRabbit

Release Notes

• New Features
  • Introduced a standalone selection service for worker selection and reservation management
  • Added HTTP API endpoints for worker lifecycle management, selection, reservation handling, and workload overlap scoring
  • Included health and readiness checks for service monitoring
  • Exposed Python CLI entry point to run the selection service via python -m dynamo.select_service

[devin-ai-integration]

Devin Review found 2 potential issues.

[coderabbitai]

Walkthrough

This PR introduces a standalone HTTP service for worker selection and reservation management in the Dynamo KV Router. The implementation includes a complete type system, error handling, in-memory worker catalog, overlap scoring utilities, an Axum-based REST API, comprehensive tests, Python CLI bindings, and integration into the E2E test framework.

Changes

Dynamo Selection Service

Layer / File(s)	Summary
Cargo features and module declarations lib/kv-router/Cargo.toml, lib/kv-router/src/services/mod.rs, lib/kv-router/src/services/selection/mod.rs, lib/kv-router/src/services/selection/README.md, lib/bindings/python/Cargo.toml	Enable standalone-selection feature and declare selection service module with conditional compilation; document service API contract, worker lifecycle states, and endpoint behaviors.
Core selection types and domain model lib/kv-router/src/services/selection/types.rs	Define SelectionKey, WorkerLifecycle, SelectionWorkerConfig, WorkerCatalogRecord with complete scheduling metadata, and request/response types for all API endpoints.
Error handling and request validation lib/kv-router/src/services/selection/error.rs, lib/kv-router/src/services/selection/input.rs	Implement SelectionError enum with HTTP status mapping, PromptRequest normalization for token/hash validation, and helpers for hash conversion.
Worker catalog storage and queries lib/kv-router/src/services/selection/catalog.rs	Implement in-memory WorkerCatalog with upsert/patch/get/list operations, lifecycle management, and selection helpers for worker discovery and routing decisions.
Overlap scoring and cache estimation lib/kv-router/src/services/selection/scoring.rs	Compute worker overlap scores from tiered match details, extract device/host-pinned/disk block counts, and derive cache-hit estimates weighted by storage tier configuration.
HTTP server and request handlers lib/kv-router/src/services/selection/server.rs	Implement Axum-based REST API with AppState management, handlers for worker lifecycle and selection/reservation operations, health checks, and graceful shutdown.
Service behavior tests lib/kv-router/src/services/selection/tests.rs	Comprehensive test suite covering readiness, worker lifecycle, selection, reservations, prefill completion, KV reconciliation, and request validation.
Python CLI and Rust bindings lib/bindings/python/Cargo.toml, lib/bindings/python/rust/lib.rs, lib/bindings/python/rust/llm/kv.rs, lib/bindings/python/src/dynamo/llm/__init__.py, lib/bindings/python/src/dynamo/select_service/*	Expose selection service as Python module via PyO3 wrapper, CLI entrypoint callable as python -m dynamo.select_service, with full module hierarchy.
E2E test framework integration tests/router/helper.py, tests/router/mocker_process.py, tests/router/test_router_e2e_with_mockers.py, tests/router/common.py	Add helpers for launching and polling selection service readiness, extend MockerProcess to manage service lifecycle and register workers, update test fixtures to verify overlap scores.

---

🎯 4 (Complex) | ⏱️ ~75 minutes

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The PR description provides a clear summary and validation instructions, but the Related Issues section does not follow the required template. The author did not complete either required path (linking to an issue or confirming no related issue with a checkbox).	Complete the Related Issues section by either specifying 'Closes #XXXX' if linked to an issue, or checking the 'Confirmed — no related issue' checkbox to fulfill the template requirement.
Docstring Coverage	⚠️ Warning	Docstring coverage is 25.96% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely describes the main change: adding a runtime-free selection service to kv-router. It is specific, avoids vague terms, and accurately reflects the primary feature introduced in the changeset.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

lib/bindings/python/rust/lib.rs (1)

254-262: ⚠️ Potential issue | 🔴 Critical | ⚡ Quick win

Missing select-service in clap error-handling feature gate.

The #[cfg(any(...))] attribute at line 255 gates clap error handling for kv-indexer and slot-tracker but omits select-service. When run_select_service (line 299) calls standalone_to_pyerr, clap errors—including --help requests—are not converted to PySystemExit; they fall through to to_pyerr and become generic PyException instances. Users running python -m dynamo.select_service --help will see an exception instead of clean help output.

🐛 Proposed fix

 fn standalone_to_pyerr(err: anyhow::Error) -> PyErr {
-    #[cfg(any(feature = "kv-indexer", feature = "slot-tracker"))]
+    #[cfg(any(feature = "kv-indexer", feature = "slot-tracker", feature = "select-service"))]
     if let Some(clap_error) = err.downcast_ref::<clap::Error>() {
         let _ = clap_error.print();
         return pyo3::exceptions::PySystemExit::new_err(clap_error.exit_code());
     }
 
     to_pyerr(err)
 }

Once this is fixed, the exception-handling workaround in lib/bindings/python/src/dynamo/select_service/main.py:14-18 can be removed.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/bindings/python/rust/lib.rs` around lines 254 - 262, The clap-specific
error handling in standalone_to_pyerr is incorrectly gated and omits the
select-service feature, so clap::Error (which yields PySystemExit for
--help/exit) falls through to to_pyerr for run_select_service; update the cfg on
the clap branch in standalone_to_pyerr to include "select-service" (i.e., use
#[cfg(any(feature = "kv-indexer", feature = "slot-tracker", feature =
"select-service"))]) so clap errors are converted to
pyo3::exceptions::PySystemExit as intended and you can remove the workaround in
run_select_service/main.py.

🧹 Nitpick comments (1)

lib/kv-router/src/services/selection/error.rs (1)

52-54: ⚡ Quick win

Fragile error detection via string matching.

The booking conflict detection relies on message.contains("already exists"), which will break if the upstream KvSchedulerError::BookingFailed message format changes. Since BookingFailed carries an opaque String, this pattern is unavoidable without refactoring the upstream error type to expose structured conflict variants.

Consider adding a comment documenting this coupling, or file a follow-up issue to refactor KvSchedulerError::BookingFailed into separate structured variants (e.g., BookingConflict, BookingOtherFailure) if the distinction is important.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/kv-router/src/services/selection/error.rs` around lines 52 - 54, The
match arm on KvSchedulerError::BookingFailed currently detects conflicts by
string matching via message.contains("already exists"), which is fragile; update
the code by adding an explicit comment above the KvSchedulerError::BookingFailed
match (referencing KvSchedulerError::BookingFailed and the
message.contains("already exists") test) documenting the coupling to the
upstream message format and the need to refactor the upstream error into
structured variants (e.g., BookingConflict vs BookingOtherFailure) as a
follow-up; optionally create or link a follow-up issue to refactor
KvSchedulerError so future maintainers know this is a technical debt hotspot.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lib/bindings/python/src/dynamo/select_service/main.py`:
- Around line 10-19: The try/except workaround in main (function main calling
run_select_service) masks errors and should be removed once clap errors are
converted to Py exceptions by Rust; update the cfg on the standalone_to_pyerr
feature gate (the #[cfg(any(...))] that guards standalone_to_pyerr) to include
feature = "select-service" so clap help/exit become PySystemExit, then delete
the try/except block in main (the code that checks for "-h"/"--help", prints
exc, and re-raises) leaving a straight call to run_select_service(args) and
returning 0; after removing the workaround ensure no stdout print of exceptions
remains and let Python receive the proper exception converted by
standalone_to_pyerr.

---

Outside diff comments:
In `@lib/bindings/python/rust/lib.rs`:
- Around line 254-262: The clap-specific error handling in standalone_to_pyerr
is incorrectly gated and omits the select-service feature, so clap::Error (which
yields PySystemExit for --help/exit) falls through to to_pyerr for
run_select_service; update the cfg on the clap branch in standalone_to_pyerr to
include "select-service" (i.e., use #[cfg(any(feature = "kv-indexer", feature =
"slot-tracker", feature = "select-service"))]) so clap errors are converted to
pyo3::exceptions::PySystemExit as intended and you can remove the workaround in
run_select_service/main.py.

---

Nitpick comments:
In `@lib/kv-router/src/services/selection/error.rs`:
- Around line 52-54: The match arm on KvSchedulerError::BookingFailed currently
detects conflicts by string matching via message.contains("already exists"),
which is fragile; update the code by adding an explicit comment above the
KvSchedulerError::BookingFailed match (referencing
KvSchedulerError::BookingFailed and the message.contains("already exists") test)
documenting the coupling to the upstream message format and the need to refactor
the upstream error into structured variants (e.g., BookingConflict vs
BookingOtherFailure) as a follow-up; optionally create or link a follow-up issue
to refactor KvSchedulerError so future maintainers know this is a technical debt
hotspot.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 58e1a8e6-3687-452f-8887-b3d1aba76083

📥 Commits

Reviewing files that changed from the base of the PR and between 9322f28 and 2a491df.

📒 Files selected for processing (22)

• lib/bindings/python/Cargo.toml
• lib/bindings/python/rust/lib.rs
• lib/bindings/python/rust/llm/kv.rs
• lib/bindings/python/src/dynamo/llm/__init__.py
• lib/bindings/python/src/dynamo/select_service/__init__.py
• lib/bindings/python/src/dynamo/select_service/__main__.py
• lib/bindings/python/src/dynamo/select_service/main.py
• lib/kv-router/Cargo.toml
• lib/kv-router/src/services/mod.rs
• lib/kv-router/src/services/selection/README.md
• lib/kv-router/src/services/selection/catalog.rs
• lib/kv-router/src/services/selection/error.rs
• lib/kv-router/src/services/selection/input.rs
• lib/kv-router/src/services/selection/mod.rs
• lib/kv-router/src/services/selection/scoring.rs
• lib/kv-router/src/services/selection/server.rs
• lib/kv-router/src/services/selection/tests.rs
• lib/kv-router/src/services/selection/types.rs
• tests/router/common.py
• tests/router/helper.py
• tests/router/mocker_process.py
• tests/router/test_router_e2e_with_mockers.py