refactor(complexity_router): drop the tier-rubric override, close the rubric on the window it was given

[tin-berri]

TLDR

Problem this solves:

• classifier_tier_rubric shipped in feat(complexity_router): let the classifier see assistant turns and rate what a short reply approves #35471 alongside the assistant-turn context window, but the two answer different halves of the same report and only the context window was asked for; the override carried a composed prompt, an overridable and a non-overridable half, a blank-is-unset rule, a length-warning validator and a pair of dashboard controls
• The rubric closed on one static line whatever classifier_context_window_size was set to, so a deployment sending no conversation was told to weigh what a short reply approves, and a deployment sending a window was told to disregard it

How it solves it:

• Removes the override and everything that existed only to support it
• Closes the rubric on one of two lines, chosen by the window: at 0 the original line byte for byte, above 0 a line that points at the quoted turns

Relevant issues

• Drops the classifier_tier_rubric operator override from ComplexityRouterConfig
• Sends the original pre-feat(complexity_router): let the classifier see assistant turns and rate what a short reply approves #35471 prompt, unchanged, whenever no conversation is quoted
• Keys the choice on classifier_context_window_size rather than on the assistant-turns toggle, so the default deployment keeps the fix

Folds in #35508, closed in favour of this. Follow-up to #35471; the dashboard half is #35500

Linear ticket

Resolves LIT-5100

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

There are exactly two prompts. With the window at 0 the classifier receives the original pre-#35471 rubric, byte for byte:

window=0 IDENTICAL to the pre-#35471 prompt: True

The two closing lines, which is the whole diff to the prompt:

classifier_context_window_size	closing line
0	Classify only the current message; use the other sections to disambiguate its difficulty.
above 0	Classify the current message, using the earlier turns quoted above it as context: when it is a short reply such as "yes" or "continue", rate the work it approves rather than the reply itself.

Everything before that line is identical in both, and identical to what ships today. The user payload is untouched, so with the window on the classifier still reads Recent conversation ... then Classify this message: yes., and with assistant turns on those quoted turns still carry user: and assistant: labels.

The end-to-end tier flip was proven against a live proxy in #35471 and is unaffected: nothing here touches extraction, the window, the payload or routing.

Type

🧹 Refactoring

Changes

• complexity_router.py drops _CLASSIFICATION_TRUST_BOUNDARY and the override composer, and _classification_system_prompt(context_window_size) now appends one of _CLASSIFICATION_CURRENT_MESSAGE_ONLY or _CLASSIFICATION_WITH_CONVERSATION
• config.py drops classifier_tier_rubric, CLASSIFIER_TIER_RUBRIC_WARN_CHARS, _warn_on_long_tier_rubric, and the logger import that existed only for that validator
• Tests lose the override's cases and gain the window-0 and window-on wording pins, plus one asserting the prompt does not vary with the assistant-turns toggle

Things a reviewer will ask about:

Why the window and not the assistant toggle. classifier_context_window_size defaults to 3, so quoting prior turns is on for every LLM-classifier deployment; classifier_context_include_assistant_turns defaults to false. The reported misclassification happens at window 3 with assistant turns off, so keying the wording on the assistant toggle would put that exact configuration back on the line that caused it. Whether the quoted turns are the user's alone or include the assistant's does not change what the model needs told, and whose turn is whose is already on the turns.

Why two static strings rather than a formatted count. The window size is not named in the prompt, so there are exactly two system-role variants, both cacheable, and no per-N string to keep true.

Existing configs. ComplexityRouterConfig is extra="allow", so a config still carrying a classifier_tier_rubric key keeps loading and the key is ignored rather than erroring on startup.

QA runbook

1. pytest tests/test_litellm/router_strategy/test_complexity_router.py, 252 passed with 14 pre-existing failures that need semantic_router installed and are red on base
2. grep -rn "tier_rubric" litellm/ tests/ returns nothing
3. Start a proxy with classifier_type: llm and classifier_context_window_size: 0, and confirm the classifier's system message ends on "Classify only the current message; use the other sections to disambiguate its difficulty."
4. Set the window to 3 and confirm it ends on the context line instead, and that flipping classifier_context_include_assistant_turns does not change the system message at all
5. Add a stale classifier_tier_rubric key and confirm the proxy still boots and ignores it

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

---

Note

Medium Risk
Changes LLM classifier instructions for all classifier_type: llm deployments, which can shift tier routing and spend; configs with stale classifier_tier_rubric are ignored but still load.

Overview
Removes the classifier_tier_rubric operator override (and its length warning validator) so the LLM classifier always uses the built-in tier definitions.

_classification_system_prompt now takes classifier_context_window_size instead of a custom rubric. It appends one of two fixed closing lines: with window 0, the pre-context wording (“classify only the current message”); with window > 0, wording that tells the model to use quoted prior turns and to rate short replies like “yes” on the work they approve—not the reply alone. The choice is keyed on window size, not the assistant-turns toggle, so default deployments keep the fix.

Tests drop override/rubric-warning cases and pin the window-dependent system prompt behavior.

^{Reviewed by Cursor Bugbot for commit 8e1b41f. Bugbot is set up for automated code reviews on this repo. Configure here.}

[tin-berri]

@greptileai can you reply here with a score and why that score?

[tin-berri]

bugbot run

[greptile-apps]

5/5

This is a clean, well-reasoned refactoring with a net negative diff (49 insertions / 138 deletions).

Why 5:

1. Correct motivation. The classifier_tier_rubric override introduced a second answer to a problem that was already solved by the context window fix in feat(complexity_router): let the classifier see assistant turns and rate what a short reply approves #35471. Removing it is the right call, not just preference.

2. No regressions. The operative rubric sentence (in the context of the conversation it continues) that fixed the "yes" misclassification is preserved verbatim in _CLASSIFICATION_SYSTEM_RUBRIC. test_rubric_rates_the_work_a_short_reply_approves would catch any future revert.

3. Backward compat handled correctly. Existing configs carrying classifier_tier_rubric silently ignore it via extra="allow" on ComplexityRouterConfig — no schema breakage, no error on load, no user-visible change.

4. Header change is appropriately scoped. The include_assistant conditional touches only the opt-in path (label_roles / include_assistant_turns), leaving the default-mode prompt byte-identical to what feat(complexity_router): let the classifier see assistant turns and rate what a short reply approves #35471 ships. The new test pins both branches.

5. Security posture improved. There is no longer a configurable path into the classifier's system role, which eliminates the class of concern the old _CLASSIFICATION_TRUST_BOUNDARY split was designed to contain. Simpler is safer here.

No issues to raise.

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 025f003. Configure here.}

[greptile-apps]

Greptile Summary

This PR removes the recently introduced operator-controlled classifier rubric and restores a single repository-owned classification prompt. It also clarifies the quoted-conversation header when assistant turns are enabled.

• Removes the classifier_tier_rubric field, composition helper, length warning, and associated tests.
• Keeps the context-aware rubric wording and trust-boundary behavior.
• Labels assistant-inclusive conversation context explicitly and tests both configuration modes.

Confidence Score: 5/5

The PR appears safe to merge, with no concrete changed-code failure identified.

The classifier continues to send a fixed trusted system rubric, legacy override keys retain the documented permissive loading behavior, and the assistant-context header follows the same configuration flag that controls role labeling and assistant-turn eligibility.

Important Files Changed

Filename	Overview
litellm/router_strategy/complexity_router/complexity_router.py	Consolidates the classifier rubric into one constant and conditionally clarifies the conversation header without introducing an actionable defect.
litellm/router_strategy/complexity_router/config.py	Removes the rubric override and its validator; permissive extra-field handling preserves loading of legacy configurations as explicitly intended.
tests/test_litellm/router_strategy/test_complexity_router.py	Removes tests for deleted override behavior while retaining end-to-end trust-boundary coverage and adding assertions for both header modes.

_{Reviews (1): Last reviewed commit: "refactor(complexity_router): drop the op..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[tin-berri]

Narrowed on 4f5e075 to a pure removal. The quoted-conversation header change I had included is reverted, so this PR now touches no prompt text at all: the composed default and the restored single constant hash identically (07d59e1d28f43648), and the trust-boundary paragraph appears on both a minus and a plus line only because it moves back inside the constant.

@greptileai can you reply here with a score and why that score?

[tin-berri]

bugbot run

[greptile-apps]

Score: 5/5

Reasons:

1. Byte-identical prompt — the PR proves the composed default and the restored single constant hash identically (07d59e1d28f43648). The trust-boundary paragraph appears on both a - and + line only because it moves back inside _CLASSIFICATION_SYSTEM_RUBRIC, not because any text changes. No behaviour regression is possible.

2. Clean deletion — 136 deletions, 9 insertions. Every removed line (classifier_tier_rubric, CLASSIFIER_TIER_RUBRIC_WARN_CHARS, _warn_on_long_tier_rubric, _classification_system_prompt, the logger import) is dead weight once the override is gone.

3. Backwards-compatible config loading — extra="allow" on ComplexityRouterConfig means existing configs with a stale classifier_tier_rubric key silently ignore it rather than failing startup. No flag needed; the model's own policy handles it.

4. Test integrity preserved — test_rubric_rates_the_work_a_short_reply_approves and test_caller_text_never_reaches_the_classifier_system_role both survive and now import the constant directly, which is stricter. The removed test cases covered the override's own logic, so dropping them is correct, not a coverage regression.

5. No security surface changes — the trust boundary paragraph is still present, unchanged, in _CLASSIFICATION_SYSTEM_RUBRIC. The removal actually tightens the security model by eliminating the operator-configurable path that could have been misused to omit the trust boundary.

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 4f5e075. Configure here.}

[tin-berri]

@greptileai can you reply here with a score and why that score?

[tin-berri]

bugbot run

[greptile-apps]

4/5

The refactor is clean, well-reasoned, and the test suite properly pins the wording. Here's the breakdown:

What's solid (pushes toward 5):

• The core logic is correct: context_window_size > 0 is the right discriminant — keying on the assistant-turns toggle instead would leave the default deployment (window=3, assistant turns off) on the wrong sentence
• Two static strings means the system role is prompt-cacheable across the session, which is an explicit win over a formatted-count alternative
• extra="allow" absorbs stale classifier_tier_rubric keys without a startup error — backward compat story is handled
• Tests pin the actual wording bytes, not just behavior, which is the right level of coverage for a prompt regression

What prevents a 5:

1. Silent drop of classifier_tier_rubric — there's no verbose_router_logger.warning when a config key is present but ignored. An operator who set a custom rubric in feat(complexity_router): let the classifier see assistant turns and rate what a short reply approves #35471 will silently get the built-in one. A single log line on config load (analogous to the removed _warn_on_long_tier_rubric) would make the transition discoverable.

2. Duplicate import in test — DEFAULT_CLASSIFIER_CONTEXT_WINDOW_SIZE is added at the module-level import (line 27-30 of the diff) and also inserted inside test_default_config_not_mutated's local import block (line 746). The local one is redundant and the indentation mismatch with ComplexityRouterConfig, is cosmetically odd (4 spaces vs 12 spaces in the same parens), though it parses fine.

Neither is a blocker. The silent-drop concern is the more substantive one, but given extra="allow" was an explicit design choice and the PR description acknowledges it, a reviewer can accept the tradeoff.

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 8e1b41f. Configure here.}

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_lit5080_drop_tier_rubric_override (8e1b41f) with litellm_internal_staging (b1fd20f)}