[ishandhanani]

Part of #16543 (PR 5 of 5)

Summary

Unify classifier worker metadata and input sequence length naming with worker selection, and deprecate the absolute prefill busy thresholds.

Classifier inputs (breaking)

The classifier retains ClassifyRequest for its per-request object and RequestClassifierContext for its router-scoped installation context.

• RequestClassifierContext::workers() returns the WorkerInfo worker selection reads (refactor(router): judge queue capacity on the shared LOAD row #16617), which also carries max_num_batched_tokens. RequestClassifierWorker is removed.
• The request's input sequence length is isl_tokens() in both plugin APIs, matching SchedulingRequest::isl_tokens and the policy YAML's ISL buckets. WorkerSelectionContext::prompt_tokens() and ClassifyRequest::input_tokens() returned the same value under two names.
• This breaks out-of-tree classifiers and selection plugins that use the removed worker type or old token accessor names, without aliases: the plugin API is not shipped yet. ThunderAgent, the default scorer, the llm router, tests, and both plugin guides are updated.

Absolute prefill thresholds (deprecated)

The absolute prefill thresholds duplicate the fractional ones, and a worker that does not report max_num_batched_tokens silently gets a 10,000,000-token default under them. Deprecated for removal in v1.7:

• policy YAML prefill_busy_threshold (logs a warning per class at load);
• --active-prefill-tokens-threshold / DYN_ACTIVE_PREFILL_TOKENS_THRESHOLD (logs a warning at startup from build_router_config(), so the frontend and any worker advertising the value each warn once; a worker without --router-mode advertises nothing and stays quiet).

Use the _frac forms instead. Docs updated.

The router policy docs now also state that a class with any queue limit set to 0 never queues: it admits while an eligible worker is below the class's busy thresholds, and rejects with 529 when every eligible worker is busy (zero_class_queue_limit_never_queues covers this). That is how a class fails fast when its workers are full, so the DEP's on_full: reject setting is not added: capacity-based rejection stays router-wide (/busy_threshold and the frontend flags), and classes wait.

Validation

After moving the deprecation warning into build_router_config() (review follow-up):

• test_router_advertisement.py and test_kv_router_args.py: 99 passed, 3 skipped (backends not installed); the two new worker CLI/env cases fail without the fix.
• mocker/tests/unit/test_config.py, frontend/tests/test_batch_gateway_args.py, frontend/tests/test_batch_gateway_transport.py: 39 passed.
• pre-commit on the touched files: clean.

After restoring ClassifyRequest:

• cargo fmt --all --check and git diff --check: clean.
• cargo test --locked -p dynamo-kv-router --lib --test plugin_api_compat: 1127 unit tests and 1 plugin API compatibility test passed; 3 ignored.
• cargo test --locked -p dynamo-custom-policy-builtin: 86 tests passed; 2 ignored.
• cargo check --locked -p dynamo-llm --no-default-features --tests: passed.
• Targeted docs lint and asset-path checks: passed (5 existing navigation warnings on other pages).

Earlier stack validation (before restoring the type name):

• cargo fmt --all --check and cargo clippy --workspace --all-targets -- -D warnings: clean.
• cargo test -p dynamo-kv-router --lib: 1120 passed; plugin_api_compat: passed.
• cargo test -p dynamo-custom-policy-builtin: passed.
• cargo test -p dynamo-mocker --lib: 272 passed.
• cargo test -p dynamo-llm --lib -- kv_router discovery::worker_monitor: 489 passed (including classifier_context_tracks_capacity_and_registered_ranks, which checks max_num_batched_tokens).
• test_kv_router_args.py: the absolute CLI flag warns; the other flags do not. 2 ais_perf_moe cases fail identically on main (local aisimulate import issue).
• Docs hooks (codespell, trailing whitespace, Fern asset paths) on the touched pages: clean.

Stack

1. feat(router): expose worker inputs to request classifiers #16616 feat(router): expose worker inputs to request classifiers
2. refactor(router): judge queue capacity on the shared LOAD row #16617 refactor(router): judge queue capacity on the shared LOAD row
3. feat(router): add a decode busy threshold to policy-class queues #16618 feat(router): add a decode busy threshold to policy-class queues
4. feat(router): enforce busy-reject thresholds in the KV router queue #16619 feat(router): enforce busy-reject thresholds in the KV router queue
5. refactor(router): unify classifier inputs; deprecate absolute busy thresholds #16620 refactor(router): unify classifier inputs; deprecate absolute busy thresholds (this PR)

[github-actions]

🌿 Fern Docs Preview: https://nvidia-preview-01a185d7-6788-7324-8ff9-31eb0cbf1750.docs.buildwithfern.com/dynamo/dev

[devin-ai-integration]

✅ Dynamo PR CI passed — run 40457088307 (attempt 1) on 508ed4968c

Gate checks: ✅ backend-status-check · ✅ deploy-status-check · ✅ dynamo-status-check

_{Posted automatically by Devin for run 40457088307. Updated on every full-CI run of this PR.}

[ishandhanani]

Deprecation only. Warnings are structured, the flag help and every docs page that names the flag are updated, and the "zero queue limit never queues" paragraph matches the existing zero_class_queue_limit_never_queues test. One nit inline.

[jthomson04]

Context naming verified on 5df9abc.

[kthui]

P3 — Low priority. The new warning is placed in RouterConfigBase.validate_rejection_thresholds(), whose only production caller is frontend validation. However, the same CLI flag and environment variable are also accepted by the worker-side router advertisement parser.

For example, a worker configured with:

--router-mode kv --active-prefill-tokens-threshold 123

passes through parse_worker_router_config() and build_router_config() without invoking that validator. The absolute value is forwarded into the advertised router configuration. The concrete vLLM path calls the parser in args.py and builds the advertisement in main.py; TRT-LLM and SGLang use the same shared functions. The Rust binding constructor also accepts the value without emitting this warning.

Consequence: operators configuring the deprecated setting through worker advertisements miss the promised startup migration notice. This is a gap in the new deprecation behavior, not evidence of incorrect routing or rejection today.

Recommended correction: cover active worker advertisements in the shared configuration path, preserving the frontend's single warning and the current unset behavior. Add coverage for both worker CLI and environment use.

Non-blocking because the absolute setting remains accepted and enforced.