feat(router): expose conditional disagg configuration

[karen-sy]

Important

This PR is [5/5] in the conditional-disaggregation stack.
Merge order is bottom-up: #11718 merges to main first, and #11723 merges last.
Do not merge this PR until every earlier PR in the checklist has merged and this branch has been rebased/retargeted as needed.

Link to the DEP: #11514

Stack merge order:

• feat(router): add conditional disagg policy kernel #11718 — feat(router): add conditional disagg policy kernel
• feat(backends): handle conditional disagg bypass annotation #11719 — feat(backends): handle conditional disagg bypass annotation
• feat(router): add decode affinity scoring #11720 — feat(router): add decode affinity scoring
• feat(router): orchestrate conditional disagg bypass #11725 — feat(router): orchestrate conditional disagg bypass
• feat(router): expose conditional disagg configuration #11723 — feat(router): expose conditional disagg configuration

Reference to original (closed) combined PR, with addressed review comments: ai-dynamo/dynamo#11357

Overview:

Details:

Where should the reviewer start?

Related Issues

⚠️ This section is required. Choose one path below and delete the other.

🔗 This PR is linked to an issue:

• Closes #XXXX

🚫 This PR is NOT linked to an issue:

• Confirmed — no related issue

Summary by CodeRabbit

• New Features

  • Added experimental conditional disaggregation for KV-aware routing.
  • Supports configurable routing policies, sequence-length thresholds, and prefill/decode load thresholds.
  • Added Python configuration support with documented defaults.

• Bug Fixes

  • Added validation for unsupported settings, malformed configuration, invalid values, and required KV-event options.

• Documentation

  • Documented setup, supported policies, backend requirements, tuning guidance, and configuration options.
  • Marked TensorRT-LLM conditional disaggregation as supported.

[github-actions]

🌿 Fern Docs Preview: https://nvidia-preview-01a0015f-716d-746a-936a-0375c86e360e.docs.buildwithfern.com/dynamo/dev

[devin-ai-integration]

Devin Review found 2 potential issues.

[MatejKosec]

Two small logging consistency issues (details in inline comments).

[MatejKosec]

reviewed with comments

[coderabbitai]

Walkthrough

The PR adds conditional disaggregation configuration parsing, validation, defaulting, CLI integration, frontend constraints, standalone-router rejection, Python/Rust propagation, tests, and documentation.

Changes

Conditional disaggregation

Layer / File(s)	Summary
Configuration parsing and application components/src/dynamo/common/configuration/groups/kv_router_args.py, components/src/dynamo/common/tests/configuration/test_kv_router_args.py	Adds conditional disaggregation fields, JSON validation, CLI options, threshold defaulting, router-kwargs lowering, and coverage for valid and invalid configurations.
Frontend validation constraints components/src/dynamo/frontend/frontend_args.py	Validates policies and thresholds and requires KV router mode with KV events when conditional disaggregation is enabled.
Standalone router rejection components/src/dynamo/router/args.py, components/src/dynamo/router/tests/test_standalone_router.py	Rejects conditional disaggregation settings in standalone router mode and tests both supported flags.
Python and Rust router bindings lib/bindings/python/rust/llm/entrypoint.rs, lib/bindings/python/src/dynamo/_core.pyi	Adds conditional disaggregation constructor parameters, Rust policy parsing, stored configuration fields, defaults, and type documentation.
Conditional disaggregation documentation docs/fern/pages/developer-guide/knowledge-base/modular-components/router/*, docs/fern/pages/developer-guide/knowledge-base/modular-components/backends/tensorrt-llm/overview.md, docs/fern/pages/reference/general/glossary.md	Documents support status, routing behavior, backend requirements, configuration fields, policies, defaults, and tuning guidance.

Estimated code review effort: 3 (Moderate) | ~30 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description provides stack context and references, but the required Overview, Details, reviewer-start, and Related Issues sections remain incomplete.	Complete the required sections and replace the placeholder issue choice with a valid issue reference or confirmation that no related issue exists.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly and concisely identifies the main change: exposing conditional disaggregation configuration in the router.

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 6

🧹 Nitpick comments (1)

lib/bindings/python/src/dynamo/_core.pyi (1)

1811-1816: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Document the activation prerequisites.

The docstring describes the policies and thresholds, but it does not state the router-mode and KV-event requirements enforced by the surrounding validation paths. Add the exact prerequisites, or state that incompatible combinations are rejected during validation.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/bindings/python/src/dynamo/_core.pyi` around lines 1811 - 1816, Update
the parameter documentation near conditional_disagg_enabled and the related
policy/threshold options to state the required router mode and KV-event
configuration for activation, matching the surrounding validation behavior.
Explicitly document that incompatible combinations are rejected during
validation, and preserve the existing descriptions and defaults.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@components/src/dynamo/common/configuration/groups/kv_router_args.py`:
- Around line 135-173: Update _parse_conditional_disagg_config in
components/src/dynamo/common/configuration/groups/kv_router_args.py (lines
135-173) so eff_isl_ratio_threshold rejects explicit null while preserving null
acceptance for the genuinely nullable thresholds. Add the corresponding invalid
JSON case to test_conditional_disagg_config_rejects_invalid_json in
components/src/dynamo/common/tests/configuration/test_kv_router_args.py (lines
462-489), asserting the existing “eff_isl_ratio_threshold must be a number”
validation message.
- Around line 135-173: Update _parse_conditional_disagg_config so
eff_isl_ratio_threshold is validated as a required numeric field, rejecting
explicit None just like eff_isl_threshold. Keep prefill_busy_threshold and
decode_busy_threshold in the nullable validation loop, since they remain
Optional.

In `@components/src/dynamo/frontend/frontend_args.py`:
- Around line 163-181: Update validate() to add non-negative checks for
conditional_disagg_prefill_busy_threshold and
conditional_disagg_decode_busy_threshold, matching the existing checks for
conditional_disagg_eff_isl_threshold. Raise ValueError with clear
option-specific messages when either threshold is below zero, while preserving
the current validation flow.
- Around line 163-181: Add a startup-time compatibility check for conditional
disaggregation in the SGLang serving initialization, rejecting configurations
that would send the bypass annotation to the SGLang decode handler before
accepting requests. Do not rely solely on FrontendConfig.validate(); use the
SGLang backend startup path or pass its capabilities into validation, while
preserving existing validation for other configurations.

In `@components/src/dynamo/router/tests/test_standalone_router.py`:
- Around line 94-96: Update the match pattern in
test_standalone_router_rejects_conditional_disagg so the period in “standalone
dynamo.router” is treated literally, using re.escape() or an explicitly escaped
dot while preserving the expected ValueError assertion.

In
`@docs/fern/pages/developer-guide/knowledge-base/modular-components/router/configuration-and-tuning.md`:
- Line 29: Update the Conditional Disaggregation link in the router
configuration documentation to target the existing disaggregated-serving.md
page, preserving the current section anchor and link text.

---

Nitpick comments:
In `@lib/bindings/python/src/dynamo/_core.pyi`:
- Around line 1811-1816: Update the parameter documentation near
conditional_disagg_enabled and the related policy/threshold options to state the
required router mode and KV-event configuration for activation, matching the
surrounding validation behavior. Explicitly document that incompatible
combinations are rejected during validation, and preserve the existing
descriptions and defaults.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: baaacd8e-6cd7-4674-b78d-2452701a8918

📥 Commits

Reviewing files that changed from the base of the PR and between abd65c9 and 63ca8ce.

📒 Files selected for processing (11)

• components/src/dynamo/common/configuration/groups/kv_router_args.py
• components/src/dynamo/common/tests/configuration/test_kv_router_args.py
• components/src/dynamo/frontend/frontend_args.py
• components/src/dynamo/router/args.py
• components/src/dynamo/router/tests/test_standalone_router.py
• docs/fern/pages/developer-guide/knowledge-base/modular-components/backends/tensorrt-llm/overview.md
• docs/fern/pages/developer-guide/knowledge-base/modular-components/router/configuration-and-tuning.md
• docs/fern/pages/developer-guide/knowledge-base/modular-components/router/disaggregated-serving.md
• docs/fern/pages/reference/general/glossary.md
• lib/bindings/python/rust/llm/entrypoint.rs
• lib/bindings/python/src/dynamo/_core.pyi