feat(router): ship ThunderAgent as a builtin plugin

docs/fern/pages/developer-guide/knowledge-base/modular-components/router/configuration-and-tuning.md

@@ -83,7 +83,7 @@ its configuration even if the endpoint is unchanged.
 - `--router-prefill-load-model`: Selects the router's prompt-side load model. `none` keeps the existing static prompt load accounting. `aic` predicts one expected prefill duration per admitted request and lazily decays only the oldest active prefill request on each worker.
 - `--router-queue-threshold`: Optional queue threshold fraction for prefill token capacity. Queueing is disabled by default; setting a numeric value enables it. The router holds incoming requests in a priority queue while all eligible workers exceed `threshold * max_num_batched_tokens`, releasing them when capacity frees up. This defers dispatch rather than rejecting work, so routing decisions use the freshest load metrics at the moment a request is sent to a worker. `nvext.agent_hints.strict_priority` selects an absolute pending-queue tier, while `nvext.agent_hints.priority` adjusts ordering within the configured policy. Must be greater than or equal to 0; use `0.0` for maximum queueing sensitivity. See the SGLang note under [Tuning Guidelines](#tuning-guidelines) for caveats around how `max_num_batched_tokens` is populated on that backend, and see [Priority Scheduling](../../../../use-cases/agents/priority-scheduling.md) for how router priority differs from backend engine priority.
 - `--router-queue-policy`: Scheduling policy for the router queue: `fcfs` (default) or `wspt`.
-- `--router-policy-config`: Startup-only YAML path for policy-class queues and worker-selection instances. When omitted, `--router-queue-threshold` and `--router-queue-policy` define one synthetic policy class. The equivalent environment variable is `DYN_ROUTER_POLICY_CONFIG`. See [Worker-Selection Policies](#worker-selection-policies) to select a built-in policy, and [Write Custom Routing Strategies](custom-worker-selection.mdx) for the linked-policy schema.
+- `--router-policy-config`: Startup-only YAML path for policy-class queues, request classification, and worker-selection instances. When omitted, `--router-queue-threshold` and `--router-queue-policy` define one synthetic policy class. The equivalent environment variable is `DYN_ROUTER_POLICY_CONFIG`. See [Worker-Selection Policies](#worker-selection-policies) to select a built-in policy, and [Write Custom Routing Strategies](custom-worker-selection.mdx) for the linked-policy schema.
 
 For how queue backpressure differs from candidate filtering and busy-threshold overload handling, see [Router Filtering](worker-filtering.md).
 
@@ -111,6 +111,7 @@ link no catalog and reject a configured policy type at startup.
 | Policy type | Behavior |
 |---|---|
 | `default` | Dynamo's built-in selector and cost model. Reserved; always available. |
+| `thunderagent` | Honors the paired ThunderAgent classifier's worker/rank preference, then falls back to the least-loaded eligible worker. Enable both roles for program-aware admission and repacking; see [ThunderAgent Program Scheduler](../../../../use-cases/agents/thunderagent-program-scheduler.md#native-frontend-plugin). |
 | `dynamo-two-tier-cost-fn` | Ranks on two tiers instead of one additive cost: active-request load first, then device-KV prefix overlap. Prefers the worker holding the largest prefix overlap unless load is badly imbalanced. Thresholds and selection order ported from the experimental SGLang router's `cache_aware_zmq` policy. Thresholds are tunable; the defaults reproduce it exactly. |
 
 Write the instance into the same YAML file that `--router-policy-config` already points at:

docs/fern/pages/use-cases/agents/thunderagent-program-scheduler.md

@@ -5,7 +5,89 @@ title: ThunderAgent Program Scheduler
 subtitle: Program-level scheduling with tool-boundary pause/resume on top of KV-aware routing
 ---
 
-> **Experimental — not a released component.** Run it from a source checkout, not from a `pip install ai-dynamo`. The CLI flags, session headers, and lifecycle hooks are all unstable and will change.
+**Experimental.** NVIDIA Dynamo includes a native ThunderAgent plugin for program-aware admission and worker placement. It runs inside the frontend and is enabled through router-policy YAML. It ports the scheduler from the [ThunderAgent](https://arxiv.org/abs/2602.13692) paper (Kang et al., 2026).
+
+## Native Frontend Plugin
+
+Use this path for aggregated serving through `dynamo.frontend`. It requires a Dynamo build that includes the `thunderagent` builtin plugin; older wheels and images do not recognize that type. The plugin is included in the builtin catalog without a separate ThunderAgent build feature. Admission behavior remains unchanged until the plugin is selected in YAML.
+
+Custom catalogs that already register `thunderagent` must remove that registration; duplicate type names fail startup.
+
+Save the following as `thunderagent.yaml`:
+
+```yaml
+request_classifier:
+  type: thunderagent
+worker_selection:
+  aggregated: thunderagent
+  instances:
+    - name: thunderagent
+      type: thunderagent
+```
+
+Start the frontend against your existing aggregated workers, using the same discovery configuration as the workers:
+
+```bash
+python3 -m dynamo.frontend \
+  --router-mode kv \
+  --router-policy-config thunderagent.yaml \
+  --router-session-affinity-ttl-secs 1800 \
+  --router-session-affinity-mode soft
+```
+
+Send a stable `X-Dynamo-Session-ID` on every turn of an agent's reasoning and tool-use chain. Dynamo also accepts the [native agent headers](session-ids.mdx#native-agent-headers). Requests without session context pass through admission unchanged. At the end of a session, a dedicated minimal request with `X-Dynamo-Session-Final: true` releases the retained program state; otherwise idle retention expires it.
+
+Select both plugin roles. The classifier owns the program table and delays admission when a session is busy or paused. When it releases a request, it supplies a preferred worker/rank. The worker selector honors that target if it is eligible, falling back to the least-loaded eligible candidate otherwise. Soft session affinity allows this repacking; hard affinity would constrain the request before the selector can move it. The `Sent` event records the worker actually selected.
+
+```mermaid
+flowchart LR
+    A[Request and session ID] --> B[ThunderAgent classifier]
+    B -->|Released request and preferred worker| C[Dynamo policy queue]
+    C --> D[ThunderAgent worker selector]
+    D --> E[Aggregated worker]
+    E -->|Progress and lifecycle events| B
+```
+
+The classifier uses each worker/rank's published GPU KV-block capacity and a logical token estimate from active and retained programs. It observes context growth during generation, then pauses and repacks programs between turns. This estimate is not physical KV occupancy and does not deduplicate shared prefixes or include host/offload capacity. Running generation is allowed to finish.
+
+### Native Plugin Parameters
+
+Set admission parameters under `request_classifier.parameters`. Leave worker-selection parameters empty. Unknown parameter names and invalid values fail startup.
+
+| Parameter | Default | Meaning |
+|---|---|---|
+| `pause_threshold` | `0.95` | Per-worker usage fraction that starts pausing programs. |
+| `pause_target` | `0.80` | Usage fraction the pause cycle drains toward. |
+| `resume_hysteresis` | `0.10` | Headroom below the pause threshold required for normal resume. |
+| `resume_timeout_seconds` | `1800` | Maximum deferral before forced release on a live worker. This does not reject the request. |
+| `session_retention_seconds` | `1800` | Retention period for an idle, continuing program. |
+| `scheduler_interval_seconds` | `5` | Interval between pressure and resume decisions. |
+| `acting_token_weight` | `1` | Capacity weight while the agent is doing tool work. |
+| `acting_decay_tau_seconds` | `1` | Acting-token decay half-life used for timeout fallback placement. |
+| `buffer_per_program` | `100` | Token headroom reserved for each active program. |
+| `max_tracked_requests` | `10000` | Independent limits on tracked requests and retained programs. Requests fail classification at the request limit. New programs fail at the program limit if no idle program can be evicted. |
+
+The default deferral timeout permits waits of roughly 30 minutes under pressure. Set client deadlines to match the workload's acceptable wait. Router policy classes and queues still apply after classification releases a request.
+
+Program state is local to each frontend's router instance. This is not a cluster-wide admission budget; keep a session's turns on the same frontend when using multiple replicas. The native plugin has been exercised with aggregated vLLM workers. Standalone EPP and disaggregated deployment guidance are outside this example.
+
+The native plugin uses the existing [router plugin API](../../developer-guide/knowledge-base/modular-components/router/custom-worker-selection.mdx#register-a-request-classifier). Its classifier and worker selector register together in Dynamo's builtin catalog. It does not expose the Python prototype's soft-demotion controls, priority boosts, host-cache budget, or separate status endpoints described below.
+
+### Native Plugin Logs
+
+The plugin logs its interval, deferral timeout, and tracking limit at INFO when admission is enabled. A WARN event reports the number of programs forcibly resumed after reaching the deferral timeout. These requests continue processing; they are not rejected. Timeout warnings are aggregated into one event per scheduler tick.
+
+To see scheduler state changes, enable DEBUG for the `thunderagent` log target before starting the frontend:
+
+```bash
+export DYN_LOG='info,thunderagent=debug'
+```
+
+Each changed tick reports counts of active, paused, and marked-for-pause programs, waiting requests, and tracked requests. Unchanged ticks are silent. Log snapshots use stored counts without scanning programs or requests, and log emission happens after releasing the scheduler lock.
+
+## Python Router Prototype
+
+The remaining sections describe the older standalone Python implementation. Run that prototype from a source checkout. Its CLI flags and observability endpoints differ from the native plugin above.
 
 `dynamo.thunderagent_router` is a standalone Dynamo router that schedules at the granularity of an agent run — the whole `LLM turn → tool call → next turn` loop — instead of individual requests. It wraps Dynamo's native KV router and adds a program-level scheduler with tool-boundary pause/resume on top of KV-aware routing, porting the scheduler from the [ThunderAgent](https://arxiv.org/abs/2602.13692) paper (Kang et al., 2026).
 

lib/bindings/python/Cargo.toml

@@ -23,7 +23,7 @@ crate-type = ["cdylib", "rlib"]
 
 [features]
 default = ["custom-policy"]
-# Links the worker-selection policies Dynamo ships plus the replaceable catalog selected by the
+# Links the router plugins Dynamo ships plus the replaceable catalog selected by the
 # dependency alias below. Enabled by default so a deployment can select a shipped policy through
 # router-policy YAML without rebuilding. A private image replaces only the catalog dependency, so
 # its policies are added alongside the shipped ones rather than displacing them.

lib/router-plugins/builtin/Cargo.toml

@@ -9,11 +9,17 @@ authors.workspace = true
 license.workspace = true
 homepage.workspace = true
 repository.workspace = true
-description = "Worker-selection policies Dynamo ships and always links"
+description = "Router plugins shipped with Dynamo"
 
 [dependencies]
 dynamo-kv-router = { workspace = true, features = ["standalone-selection"] }
 serde.workspace = true
+async-trait.workspace = true
+parking_lot.workspace = true
+thiserror.workspace = true
+tokio.workspace = true
+tracing.workspace = true
 
 [dev-dependencies]
 tempfile = { workspace = true }
+tokio-util = { workspace = true }

lib/router-plugins/builtin/examples/thunderagent.yaml

@@ -0,0 +1,24 @@
+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Aggregated frontend: enable soft session affinity and send stable session IDs.
+# Omitted parameters use the plugin defaults.
+request_classifier:
+  type: thunderagent
+worker_selection:
+  aggregated: thunderagent
+  instances:
+    - name: thunderagent
+      type: thunderagent

lib/router-plugins/builtin/src/lib.rs

@@ -1,7 +1,7 @@
 // SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 // SPDX-License-Identifier: Apache-2.0
 
-//! Worker-selection policies Dynamo ships.
+//! Router plugins Dynamo ships.
 //!
 //! Any build that enables the Python bindings' `custom-policy` feature links these, and that
 //! feature is on by default, so a frontend deployment selects one through router-policy YAML
@@ -12,29 +12,30 @@
 //! register before it, so a custom image's catalog adds its policies alongside these rather than
 //! displacing them.
 //!
-//! Each policy is one module here. To ship another, add a module and a line in [`register`], then
+//! Each plugin is one module here. To ship another, add a module and a line in [`register`], then
 //! add a row to the policy table in the router configuration guide. Keep a policy in a single file
-//! until it needs submodules, then promote it to a directory. If a policy ever needs a dependency
-//! beyond `dynamo-kv-router`, put it behind its own default-on Cargo feature so a build can drop
-//! it; every policy registered here is compiled into every artifact that links this crate.
+//! until it needs submodules, then promote it to a directory. Every plugin registered here is
+//! compiled into artifacts that link this crate; router-policy YAML selects which ones run.
 
+mod thunderagent;
 mod two_tier_cost_fn;
 
-use dynamo_kv_router::plugins::{RouterPluginRegistry, WorkerSelectionPolicyRegistryError};
+use dynamo_kv_router::plugins::{RouterPluginRegistry, RouterPluginRegistryError};
 
 /// Register every policy Dynamo ships.
 ///
 /// `default` is reserved by the registry for Dynamo's built-in worker selector, so no policy here
 /// can shadow it. A later catalog that reuses one of these type names fails registration rather
 /// than overriding it.
-pub fn register(
-    registry: &mut RouterPluginRegistry,
-) -> Result<(), WorkerSelectionPolicyRegistryError> {
-    two_tier_cost_fn::register(registry)
+pub fn register(registry: &mut RouterPluginRegistry) -> Result<(), RouterPluginRegistryError> {
+    two_tier_cost_fn::register(registry)?;
+    thunderagent::register(registry)?;
+    Ok(())
 }
 
 #[cfg(test)]
 mod tests {
+    use dynamo_kv_router::plugins::WorkerSelectionPolicyRegistryError;
     use dynamo_kv_router::plugins::worker_selection::WorkerSelectionPolicyFactory;
     use dynamo_kv_router::{KvRouterConfig, RoutingPartitionRef, WorkerType};