ai-dynamo · PeaBrane · Jun 16, 2026 · Jun 16, 2026 · Jun 16, 2026 · Jun 16, 2026
@@ -64,5 +64,6 @@ For basic model registration without KV routing, use `--router-mode round-robin`
 - **[Router Examples](router-examples.md)**: Python API usage, K8s examples, and custom routing patterns
 - **[Router Testing](router-testing.md)**: Test layers from Rust unit tests to fixture-backed replay and full process E2E
 - **[Standalone Indexer](standalone-indexer.md)**: Run the KV indexer as a separate service for independent scaling
+- **[Standalone Selection Service](standalone-selection.md)**: Expose KV-aware selection and reservation accounting over HTTP
 - **[Standalone Slot Tracker](standalone-slot-tracker.md)**: Run active-request load accounting as a separate HTTP service
 - **[Router Design](../../design-docs/router-design.md)**: Architecture details, algorithms, and event transport modes
@@ -179,4 +179,6 @@ Disaggregated mode is activated automatically when prefill workers register alon
 - **[Router Examples](router-examples.md)**: Python API usage, K8s examples, and custom routing patterns
 - **[Router Testing](router-testing.md)**: Recommended test layers for non-trivial router changes
 - **[Standalone Indexer](standalone-indexer.md)**: Run the KV indexer as a separate service
+- **[Standalone Selection Service](standalone-selection.md)**: Select workers and account for reservations without forwarding requests
+- **[Standalone Slot Tracker](standalone-slot-tracker.md)**: Run active-request accounting as a separate service
 - **[KV Event Replay — Dynamo vs vLLM](kv-event-replay-comparison.md)**: Gap detection and replay behavior
@@ -0,0 +1,268 @@
+---
+# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+title: Standalone Selection Service
+subtitle: Select workers and account for reservations without forwarding inference requests
+---
+
+## Overview
+
+The standalone selection service (`python -m dynamo.select_service`) exposes the
+KV router's worker selection and active-load accounting over HTTP. It does not
+forward model requests or own response streams. External runtimes such as Ray
+register their worker catalog, request a selection, contact the selected worker,
+and report the reservation lifecycle.
+
+The service combines:
+
+- KV overlap indexing from worker ZMQ events.
+- KV-aware and load-aware worker selection.
+- Explicit or atomic selection and reservation.
+- Best-effort active-load synchronization between selector replicas.
+- Startup KV index recovery from another selector or standalone indexer.
+
+## Build And Launch
+
+Build the Python bindings with the `select-service` feature:
+
+```bash
+cd lib/bindings/python
+VIRTUAL_ENV=../../../.venv ../../../.venv/bin/maturin develop --uv --features select-service
+```
+
+Launch the service from the repository root:
+
+```bash
+.venv/bin/python -m dynamo.select_service --port 8092
+```
+
+The service binds to `0.0.0.0` and does not provide authentication. Run it on a
+trusted internal network or place it behind an appropriate network policy.
+
+### CLI
+
+| Flag | Default | Description |
+|------|---------|-------------|
+| `--port` | `8092` | HTTP server port. |
+| `--threads` | `4` | KV indexer worker threads. |
+| `--indexer-peers` | none | Comma-separated HTTP URLs used for startup KV recovery through `/dump`. |
+| `--replica-sync-port` | none | Local ZMQ PUB port for active-load lifecycle events. The selector binds `tcp://*:<port>` internally. |
+| `--replica-sync-peers` | none | Comma-separated ZMQ PUB endpoints for selector peers. Requires `--replica-sync-port`. |
+
+Router scheduling behavior continues to use the standard Dynamo router
+environment configuration.
+
+## Worker Registration
+
+Every selector replica must receive the same worker catalog before it serves
+selection traffic. Replica traffic never creates workers.
+
+```http
+POST /workers
+Content-Type: application/json
+
+{
+  "worker_id": 1,
+  "model_name": "model",
+  "tenant_id": "default",
+  "endpoint": "http://worker:8000",
+  "block_size": 16,
+  "data_parallel_start_rank": 0,
+  "data_parallel_size": 2,
+  "kv_events_endpoints": {
+    "0": "tcp://worker:5557",
+    "1": "tcp://worker:5558"
+  },
+  "replay_endpoint": "tcp://worker:5560"
+}
+```
+
+`POST /workers` returns `201`. `PATCH /workers/{worker_id}` updates supplied
+fields, `DELETE /workers/{worker_id}` removes the worker, and `GET /workers`
+lists catalog state. `model_name` and `tenant_id` scope all selection, indexer,
+and load state; both default to `"default"` when omitted.
+
+`GET /health` is process liveness. `GET /ready` returns `200` only after at
+least one worker is schedulable, otherwise `503` with lifecycle details.
+
+## Selection API
+
+### `POST /select`
+
+Select a worker without booking active load:
+
+```json
+{
+  "selection_id": "select-123",
+  "model_name": "model",
+  "tenant_id": "default",
+  "block_hashes": [11, 12, 13, 14, 15, 16, 17, 18],
+  "sequence_hashes": [21, 22, 23, 24, 25, 26, 27, 28],
+  "isl_tokens": 512
+}
+```
+
+### `POST /select_and_reserve`
+
+Select and atomically book load in the receiving selector process. Supply a
+globally unique `reservation_id`, or allow the service to generate one:
+
+```json
+{
+  "selection_id": "select-123",
+  "reservation_id": "request-123",
+  "model_name": "model",
+  "tenant_id": "default",
+  "block_hashes": [11, 12, 13, 14, 15, 16, 17, 18],
+  "sequence_hashes": [21, 22, 23, 24, 25, 26, 27, 28],
+  "isl_tokens": 512
+}
+```
+
+Both endpoints return the same selection shape:
+
+```json
+{
+  "selection_id": "select-123",
+  "reservation_id": "request-123",
+  "model_name": "model",
+  "tenant_id": "default",
+  "worker_id": 1,
+  "dp_rank": 0,
+  "endpoint": "http://worker:8000",
+  "block_size": 16,
+  "overlap": {
+    "longest_matched": 128,
+    "gpu": 64,
+    "dp": {"0": 64, "1": 32},
+    "cpu": 96,
+    "disk": 128
+  },
+  "effective_prefill_tokens": 384
+}
+```
+
+`selection_id` and `reservation_id` are omitted when absent. All `overlap`
+values are matched token counts. `gpu`, `cpu`, and `disk` use the cumulative
+Mooncake tier semantics documented in the standalone indexer's
+[per-instance tier breakdown](standalone-indexer.md#per-instance-tier-breakdown).
+A zero-overlap response includes the selected `dp_rank` with value `0`.
+
+The overlap summary is raw observability. `effective_prefill_tokens` is the
+authoritative weighted prefill-load value computed by the same cache-credit
+formula used for scheduler booking. It is not derived from `longest_matched`.
+
+The previous public fields `cached_tokens` and `effective_overlap_blocks` have
+been removed. Their values remain internal scheduler inputs.
+
+## Ray Select-Then-Reserve Flow
+
+Ray can keep model invocation separate from selector admission:
+
+1. Call `POST /select`.
+2. Send the request to the returned `endpoint` and `dp_rank`.
+3. Call `POST /reservations` with a globally unique reservation ID, selected
+   worker identity, the same prompt representation, and the returned
+   `effective_prefill_tokens`.
+4. Report prefill completion and request completion through the lifecycle API.
+
+```http
+POST /reservations
+Content-Type: application/json
+
+{
+  "reservation_id": "request-123",
+  "model_name": "model",
+  "tenant_id": "default",
+  "worker_id": 1,
+  "dp_rank": 0,
+  "sequence_hashes": [21, 22, 23, 24, 25, 26, 27, 28],
+  "isl_tokens": 512,
+  "effective_prefill_tokens": 384
+}
+```
+
+When supplied, `effective_prefill_tokens` is authoritative and directly enables
+prefill-load tracking. It must not exceed the normalized input sequence length.
+When omitted, existing router configuration controls prefill tracking. The
+reservation API does not accept or derive accounting from overlap fields.
+
+## Reservation Lifecycle
+
+```http
+POST /reservations/{reservation_id}/prefill_complete
+POST /reservations/{reservation_id}/output_block
+DELETE /reservations/{reservation_id}
+```
+
+`prefill_complete` clears active prefill load. `output_block` updates only the
+receiving selector's local decode-block accounting and accepts an optional
+`decay_fraction` in `[0.0, 1.0]`. `DELETE` frees the reservation.
+
+**NOTE:** Output-block updates are intentionally not replica-synchronized.
+They can occur at high frequency, and broadcasting them would consume
+disproportionate network bandwidth.
+
+## Peer Planes
+
+The selector has two independent peer configurations:
+
+| Plane | Transport | Flags | Purpose |
+|-------|-----------|-------|---------|
+| Indexer recovery | HTTP | `--indexer-peers` | Fetch a compatible `/dump` during startup and replay KV events into the local indexer. |
+| Replica synchronization | ZMQ | `--replica-sync-port`, `--replica-sync-peers` | Share admission, prefill-complete, and free events by model and tenant. |
+
+Example:
+
+```bash
+.venv/bin/python -m dynamo.select_service \
+  --port 8092 \
+  --indexer-peers http://selector-b:8092 \
+  --replica-sync-port 9092 \
+  --replica-sync-peers 'tcp://selector-b:9092'
+```
+
+Configure the reverse peer direction on selector B for bidirectional lifecycle
+synchronization. `GET /dump` exposes the selector's current indexer snapshot in
+the same recovery format as the standalone indexer.
+
+Replica-sync peers may also be changed without restarting the selector:
+
+```http
+POST /replica_sync/register_peer
+Content-Type: application/json
+
+{"endpoint":"tcp://selector-b:9092"}
+```
+
+The same body is accepted by `POST /replica_sync/deregister_peer`.
+`GET /replica_sync/peers` returns the sorted configured endpoints. Dynamic
+membership is in-memory; after restart, only peers supplied through
+`--replica-sync-peers` are restored. These routes only manage live ZMQ
+replica-sync peers. They do not alter the HTTP indexer-recovery peers.
+
+## Consistency Invariants
+
+- Replica synchronization is bounded and best-effort. Delays, reordering,
+  dropped events, and temporary active-load divergence are accepted.
+- There is no sequencing, acknowledgement, replay, backpressure, or
+  resynchronization for replica lifecycle events.
+- Unknown worker, model, tenant, DP-rank, and block-size events are dropped.
+  Register the same worker catalog on every selector before routing traffic.
+- Admission, prefill-complete, and free are synchronized. Output-block growth
+  remains local to avoid excessive network bandwidth.
+- Startup recovery waits for recovered events to be submitted to the indexer,
+  not for complete processing. Early selections may temporarily miss recovered
+  KV state.
+- `/select` followed by `/reservations` provides eventual, not atomic,
+  cross-replica admission. Use `/select_and_reserve` for atomic local booking.
+- Reservation IDs must be globally unique. Existing conflict and retry
+  semantics are unchanged; no idempotency ledger is added.
+
+## Inspection APIs
+
+- `GET /loads` returns active-load snapshots, optionally filtered by
+  `model_name` and `tenant_id`.
+- `POST /potential_loads` estimates worker load for a prompt without selection.
+- `POST /overlap_scores` returns per-worker/per-rank tiered overlap rows.
+- `GET /dump` returns the compatible indexer recovery snapshot.
@@ -40,8 +40,7 @@ Enable replica synchronization:
 ```bash
 .venv/bin/python -m dynamo.slot_tracker \
   --port 8091 \
-  --replica-sync-bind 'tcp://*:8092' \
-  --replica-sync-advertise 'tcp://slot-tracker-a:8092' \
+  --replica-sync-port 8092 \
   --replica-sync-peers 'tcp://slot-tracker-b:8092'
 ```
 
@@ -55,14 +54,14 @@ internal network or place it behind an appropriate network policy.
 
 ## Replica Synchronization
 
-`--replica-sync-bind` enables a ZMQ PUB endpoint and replica-event consumption. The
-service generates an ephemeral process identity internally; it is not a configuration
-parameter. `--replica-sync-advertise` is the externally reachable form of the local
-endpoint and prevents exact self-registration. `--replica-sync-peers` accepts a
-comma-separated list of peer PUB endpoints.
+`--replica-sync-port` enables a ZMQ PUB endpoint and replica-event consumption. The
+service binds `tcp://*:<port>` internally and generates an ephemeral process identity;
+neither the bind expression nor process identity is a configuration parameter.
+`--replica-sync-peers` accepts a comma-separated list of peer PUB endpoints and requires
+`--replica-sync-port`.
 
 Peer connections are directional. For bidirectional synchronization, configure each
-replica with the other replica's advertised endpoint. All replicas must independently
+replica with the other replica's reachable ZMQ endpoint. All replicas must independently
 receive the same worker registrations before lifecycle traffic begins. A replica event
 for an unknown `(model_name, tenant_id)`, block size, worker ID, or DP rank is dropped.
 Replica traffic never creates workers.
@@ -77,15 +76,17 @@ cross-replica lifecycle ownership.
 Peers may also be managed dynamically:
 
 ```http
-POST /register_peer
+POST /replica_sync/register_peer
 Content-Type: application/json
 
-{"url":"tcp://slot-tracker-b:8092"}
+{"endpoint":"tcp://slot-tracker-b:8092"}
 ```
 
-The same body is accepted by `POST /deregister_peer`. `GET /peers` returns the sorted
-configured endpoints. Registration confirms that the endpoint was accepted by the local
-SUB socket, not that the asynchronous ZMQ subscription handshake has completed.
+The same body is accepted by `POST /replica_sync/deregister_peer`.
+`GET /replica_sync/peers` returns the sorted configured endpoints. Registration confirms
+that the endpoint was accepted by the local SUB socket, not that the asynchronous ZMQ
+subscription handshake has completed. Dynamic membership is in-memory; after restart,
+only peers supplied through `--replica-sync-peers` are restored.
 
 ## Common Responses
 

@@ -372,6 +372,8 @@ navigation:
             path: components/router/router-examples.md
           - page: Standalone Indexer
             path: components/router/standalone-indexer.md
+          - page: Standalone Selection Service
+            path: components/router/standalone-selection.md
           - page: Standalone Slot Tracker
             path: components/router/standalone-slot-tracker.md
           - page: KV Event Replay — Dynamo vs vLLM