[ishandhanani]

Summary

PR ai-dynamo#16618 added two separate things, and this PR is about the second:

• (a) Decode check in BusyThresholds::has_room: a rank is full once the KV blocks its active requests hold exceed decode_busy_threshold_frac of its total_kv_blocks, judged on the shared LOAD row. feat(router): enforce busy-reject thresholds in the KV router queue ai-dynamo/dynamo#16619 and refactor(router): unify classifier inputs; deprecate absolute busy thresholds ai-dynamo/dynamo#16620 build on it: the router-wide --active-decode-blocks-threshold maps into this check on the reject path. (a) is correct in every topology, so this PR does not change it, the reject path, or the worker monitor.
• (b) Per-class decode wait limit: policy_classes[].decode_busy_threshold_frac in the router policy YAML, which the policy-class queue gate waits on. (b) is unreleased (not in any release branch or tag), and as merged it is correct only in aggregated serving.

This PR implements (b) so it is correct in both topologies:

• Aggregated: unchanged. The router's one queue checks both class fractions on the same workers.
• Disaggregated with KV routing: the prefill router's queue is the only admission gate. It checks the class prefill fraction on the prefill workers and the class decode fraction on the decode workers. For the decode check, it reads the decode router's booked load directly, with no message to the decode actor. When the decode router frees KV, it fires a separate broadcast signal (RoomChanged) that wakes the prefill queue. That signal does not share the decode actor's notify_one CapacityUpdates. The decode router books each decode leg that follows a remote prefill (RequestPhase::Decode with staged KV) straight away: no class wait and no deadline expiry, because the leg's prefill is already running. Router-wide reject thresholds still apply when the decode worker is selected.

Why this changes behavior: disaggregated KV routing has two policy-class queues in series, prefill then decode, and both get the same policy profile. As merged, the class decode limit did nothing at the prefill router, which never tracks held blocks. At the decode router it could hold a decode leg after its prefill was already dispatched. That held prefill KV in place, and broke SGLang bootstrap, where prefill and decode must run at the same time.

Design:

• lib/kv-router: a new PairedPoolRoom hook that the host installs on a scheduler (set_paired_pool, like set_selection_trace). In each gate pass the queue checks it together with the existing prefill check, in both the drain path and the never-queue direct path. Only classes with a decode_busy_threshold_frac use it, and without a hook nothing changes. For requests without a LoRA, one answer is cached per class per gate pass. LocalScheduler::any_rank_has_room reads booked load from the slot registry with no actor message, and subscribe_room_changed exposes the broadcast hint. The hint fires on local releases, the actor's cleanup drain, peer-router state changes, and worker config changes. ScheduleRequest / SelectionOperation carry an in-process admitted_upstream flag, so the queue books those requests directly. Wire DTOs (PreprocessedRequest, RouterConfigOverride) are unchanged.
• lib/llm: the frontend now installs the decode RoutingHost on the PrefillRouter whenever routing is KV, not only under conditional disagg. Every prefill binding's KV router gets a lazily resolved DecodePoolRoom (it holds a Weak<KvRouter>). Until the decode host exists, the pool reports room, so nothing changes. RoutingHost maps a decode leg with staged_kv_cleanup (frontend-only, #[serde(skip)]) to AdmittedUpstream. A conditional-disagg bypass also has RequestPhase::Decode but has nothing staged, so the decode queue stays its gate and its behavior is unchanged.

Decode-room eligibility (what the prefill hop applies):

• Applied: decode worker reachability and the request's LoRA. Both come from reusing the decode router's existing availability filter. That filter narrows LoRA candidates by the request's pin and allowed workers, which at the prefill hop name prefill workers. So a LoRA request that also carries a prefill pin or allowed set is checked on reachability alone.
• Not applied: a decode worker pin (decode_worker_id), allowed-worker hints, routing constraints, and the KV-transfer topology domain of the chosen prefill worker.
• Decode selection still enforces exact decode eligibility (RoutingEligibility, lib/kv-router/src/scheduling/filter.rs). So when the coarse gate admits a request, it is routed at decode exactly as before feat(router): add a decode busy threshold to policy-class queues ai-dynamo/dynamo#16618.

Known limits and follow-ups:

• Between the decode-room check at the prefill hop and the decode booking (after prefill dispatch), a burst of prefill admissions can overshoot the decode fraction. Each admission sees decode room before the earlier legs have booked. A pending-decode reservation count would close this gap.
• If either hop is not KV-routed, nothing checks the class decode limit.
• Request classifiers run only on the decode/aggregated router, so a class a classifier assigns there is not known at the prefill hop. This is unchanged by this PR.

Part of ai-dynamo#16543

Validation

All commands use a per-worktree CARGO_TARGET_DIR.

• cargo fmt --all --check: clean.
• cargo clippy -p dynamo-kv-router -p dynamo-llm --all-targets -- -D warnings: pass, no warnings.
• cargo test -p dynamo-kv-router --lib: 1134 passed, 0 failed, 3 ignored.
• cargo test -p dynamo-kv-router --lib --features standalone-selection, which also runs the selection-core tests: 1393 passed, 0 failed, 3 ignored.
• cargo test -p dynamo-llm --lib kv_router::: 467 passed, 0 failed. The new end-to-end test also passed 5 of 5 repeated runs.
• cargo bench -p dynamo-kv-router --no-run: builds (4 bench targets). With --features standalone-selection,bench: builds (6 targets).
• cd lib/bindings/python && cargo check: pass.
• pre-commit run --files <changed files>, run through uvx: 11 hooks passed, the rest skipped (no matching files).
• python3 docs/fern/scripts/docs_lint.py <changed page>: 0 errors. The 5 NAV warnings are on other, pre-existing pages.

New tests:

• scheduling::queue::tests::paired_pool_holds_until_its_room_signal_releases_the_request: the request stays held while the paired pool has no room. Room alone does not release it; the pool's room signal does, with no other capacity change.
• scheduling::queue::tests::paired_pool_rejects_a_never_queue_class_like_its_own_workers: the never-queue direct path and the queue agree.
• scheduling::queue::tests::paired_pool_is_ignored_by_classes_without_a_decode_limit: a class without a decode limit never consults the hook, and its prefill limit behaves as before.
• scheduling::queue::tests::admitted_upstream_skips_class_limits_but_not_reject_thresholds: the request is booked past a full class limit and ahead of the backlog, an already-passed deadline does not expire it, and reject thresholds still reject it.
• scheduling::local::tests::pool_room_reads_booked_decode_blocks_and_signals_releases: the direct read honors the worker filter, and the room signal fires on release even with queueing disabled.
• kv_router::routing_host::tests::disagg_decode_wait_limit_gates_the_prefill_hop_and_never_the_decode_hop: two real KV hosts share one policy file. Staged decode legs book past the class decode limit. A conditional-bypass-shaped decode request still waits at decode. A new prefill request waits at the prefill hop while decode is full, and freed decode KV releases it.
• kv_router::prefill_router::tests::kv_decode_host_resolves_the_decode_pool_without_conditional_disagg.

Queue-gate benchmark for the aggregated path (selection_core no_hints_queue_gate, base 4cb0a16 vs this branch, shared host with load average 4 to 7):

workers	base	this branch
64	39.6 us	36.1 us
512	90.5 us	76.7 us
4096	614.6 us	608.0 us

No regression within noise. The aggregated gate only adds one per-class array reset and one early-returning check per predicate.