feat(complexity-router): add a bundled Nadir classifier plugin

[doramirdor]

TLDR

Problem this solves:

• classifier_type: custom ships no classifier, so every operator writes one
• Complexity routing today is a local scorer or a paid LLM call, nothing in between

How it solves it:

• Bundles a plugin that asks Nadir's /v1/bucket for the tier
• Nadir grades the request without generating an answer, so there is no classifier model to host
• Decision only: the tier's models, the provider call and your keys never leave this proxy

User Flow

Before: a proxy admin who wants a trained complexity classifier has to write and host the plugin themselves

1. They set classifier_type: custom in complexity_router_config and have nothing to point classifier_plugin at
2. Pointing it at litellm.router_strategy.complexity_router.nadir_classifier.nadir_classifier fails the proxy at startup: ImportError: Could not import nadir_classifier from litellm.router_strategy.complexity_router.nadir_classifier
3. Their remaining options are the local keyword scorer or a classifier model they pay for on every request

After: the same admin names the bundled plugin and gets trained classification with no classifier model

1. They add classifier_type: custom and classifier_plugin: litellm.router_strategy.complexity_router.nadir_classifier.nadir_classifier to complexity_router_config, set NADIR_API_KEY, and restart the proxy
2. They send POST https://litellm-domain/v1/chat/completions with "model": "smart-router" and "what is 2+2?", and it is answered by the SIMPLE tier model
3. They send the same POST with a long design-and-prove-it task, and it is answered by the COMPLEX tier model
4. https://litellm-domain/ui/?page=logs shows both rows with routing_decision.cause: classifier_plugin and the tier each request landed in
5. If Nadir is unreachable or slow, the row reads cause: heuristic_scorer instead and the completion still succeeds

Relevant issues

Docs companion: BerriAI/litellm-docs#1543 documents the plugin. CI no longer depends on it: the Nadir provider page merged in BerriAI/litellm-docs#861 already mentions NADIR_API_KEY and NADIR_API_BASE, so test_env_keys.py passes here.

Pre-Submission checklist

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Status: CI on af2f2fe: 97 passed, 1 skipped, 0 failed, and CodeQL reports no new alerts in the changed code. Greptile: 4/5 on af2f2fe. Its three first-review findings are resolved: credential scoping and the patched client are fixed in 84d907e, and it withdrew the tier_definitions one as a documented boundary. Its one open P2 asks for the Nadir HTTP handling to move under llms/; the reply in that thread points at jev_classifier.py, the router's other external classifier, which keeps the same layout. af2f2fe reads the verdict through a TypeAdapter, which clears the CodeQL mixed-returns note on classify() without tripping basedpyright's reportMatchNotExhaustive. Locally: test_nadir_classifier.py 28 passed, and ruff_strict_gate, type_discipline_gate and test_quality_gate pass with --base c19ce71bcd.

Screenshots / Proof of Fix

A real proxy, started from a checkout of each commit with the config below, with NADIR_API_KEY set and the tier models on OpenRouter. The routing decision is a live call to Nadir's /v1/bucket, and the completions are live, billed calls to the tier models. Nothing is mocked.

model_list:
  - model_name: openai/gpt-4o-mini
    litellm_params:
      model: openrouter/openai/gpt-4o-mini
      api_key: os.environ/OPENROUTER_API_KEY
  - model_name: openai/gpt-4.1
    litellm_params:
      model: openrouter/openai/gpt-4.1
      api_key: os.environ/OPENROUTER_API_KEY
  - model_name: anthropic/claude-sonnet-4.5
    litellm_params:
      model: openrouter/anthropic/claude-sonnet-4.5
      api_key: os.environ/OPENROUTER_API_KEY
  - model_name: smart-router
    litellm_params:
      model: auto_router/complexity_router
      complexity_router_config:
        classifier_type: custom
        classifier_plugin: litellm.router_strategy.complexity_router.nadir_classifier.nadir_classifier
        classifier_fallback: heuristic
        tiers:
          SIMPLE: openai/gpt-4o-mini
          MEDIUM: openai/gpt-4.1
          COMPLEX: anthropic/claude-sonnet-4.5

Before (c19ce71)

case 1: proxy startup

1. litellm --config config.yaml
2. The proxy exits during startup: ImportError: Could not import nadir_classifier from litellm.router_strategy.complexity_router.nadir_classifier

case 2: routed completions

1. Unreachable: the proxy never starts, so no request is served.

After (af2f2fe)

case 1: proxy startup

1. litellm --config config.yaml --detailed_debug
2. The proxy starts and logs ComplexityRouter initialized for smart-router with tiers: {'SIMPLE': 'openai/gpt-4o-mini', 'MEDIUM': 'openai/gpt-4.1', 'COMPLEX': 'anthropic/claude-sonnet-4.5'}. Startup builds the Router from this config, which deepcopies the plugin instance, and GET /health/liveliness answers "I'm alive!"

case 2: routed completions

1. curl -X POST http://localhost:4000/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model": "smart-router", "messages": [{"role": "user", "content": "what is 2+2?"}], "max_tokens": 64}'
2. HTTP 200, answer "2 + 2 = 4.", 14 in / 8 out, x-litellm-response-cost: 6.9e-06. Proxy log: ComplexityRouter: routing decision cause=classifier_plugin, tier=SIMPLE, score=n/a, signals=('classifier-plugin:SIMPLE',), routed_model=openai/gpt-4o-mini
3. The same request with the content "design a distributed consensus protocol with partition tolerance and prove liveness under asynchrony"
4. HTTP 200, a long design answer, 23 in / 2834 out, x-litellm-response-cost: 0.042579. Proxy log: ComplexityRouter: routing decision cause=classifier_plugin, tier=COMPLEX, score=n/a, signals=('classifier-plugin:COMPLEX',), routed_model=anthropic/claude-sonnet-4.5
5. The answer runs past max_tokens: 64 because the complexity router replaces the caller's value with the tier model's output ceiling by default (max_tokens_from_tier_model); the plugin does not touch it

case 3: unit tests

1. pytest tests/test_litellm/router_strategy/test_nadir_classifier.py -q
2. 28 passed

Type

🆕 New Feature

Caveats (if any)

• A network call per classified request. About 160ms warm from a laptop to the hosted API against under a millisecond for heuristic / heuristic_v2. Anything past classifier_plugin_timeout_ms (default 3000) routes on classifier_fallback rather than waiting, so the cost of a slow or dead classifier is a fallback, never a failed completion.
• The messages go to the configured host, which is a third party unless you self-host Nadir. Same disclosure classifier_type: llm carries with a hosted classifier model. Stated in the README section and the module docstring.
• Three tiers, not four. Nadir grades simple / medium / complex, so REASONING is never returned; keyword rules and the reasoning override still place requests there.
• Renamed tiers. A tier_labels router uses the module instance unchanged, since the router still accepts the default names it returns. Only a tier_definitions router needs its own NadirComplexityClassifier(tier_map=...), defined in a module of its own and named by dotted path. An unmapped verdict declines rather than guesses.
• Without NADIR_API_KEY the endpoint still answers, under a per-IP rate limit sized for trying the plugin out, and requests over it route on classifier_fallback. With a key, that account's request log keeps the user text of each classified request, or only a hash of it when the account has prompt storage turned off.
• One line outside the feature: a httpxSpecialProvider member, so the call reuses litellm's cached async client instead of opening a private pool.
• I maintain Nadir. The plugin is opt-in, off by default, and touches nothing unless classifier_plugin names it.

🤖 Generated with Claude Code

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing doramirdor:nadir-classifier-plugin (df0714e) with main (9a3f000)}

[greptile-apps]

Confidence Score: 4/5

[Medium risk] Adds a new request classifier plugin for the routing system.

The behavior appears safe, but the provider-code placement requirement should be satisfied before merging

Findings

1.  Provider code outside llms ▶

Summary

The PR bundles a Nadir classifier for complexity routing, documents its configuration and fallback behavior, and adds focused tests

• The documented plugin path uses LiteLLM’s shared async HTTP client
• Provider-specific Nadir handling remains in the router package rather than llms/

_{Reviews (3) · Last reviewed commit: "fix(complexity-router): read the Nadir v..."}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[doramirdor]

CI status, so the two red checks are not mistaken for code problems: 82 passing, 2 failing, 1 skipped, and both failures are the same cross-repo docs check.

tests/documentation_tests/test_env_keys.py fails any env var read under litellm/ that BerriAI/litellm-docs does not mention, and it runs in both code-quality (step documentation_test_env_keys) and Unit Tests: Documentation Validation. The plugin reads NADIR_API_KEY and NADIR_API_BASE.

The docs half is open at BerriAI/litellm-docs#1543 and is ready: two rows in the environment-variable reference plus a subsection under the existing custom classifier plugin docs, with an availability admonition naming this PR so the page reads correctly whichever merges first. With that branch checked out at docs/my-website, test_env_keys.py, test_router_settings.py and test_api_docs.py all pass against this branch.

Nothing else is outstanding on this side.

[doramirdor]

@greptileai please re-review. The three findings from the first review (credential scoping, tier_labels routers, the patched client) are addressed in 84d907e, and af2f2fe reads the verdict through a TypeAdapter so every path in classify() returns explicitly.

[doramirdor]

@greptileai please re-review. Two of the three first-review threads are fixed in 84d907e and resolved. The tier_definitions one is answered in its thread: it is a documented boundary, and a misconfigured router logs a warning and routes on its fallback.

[doramirdor]

@krrish-berri-2 while you are in the Nadir code: this one is ready for review as well. It adds a bundled Nadir classifier plugin to the complexity router (decision only, no generation call). Mergeable against main, CI green, Greptile 4/5.

[doramirdor]

@tin-berri @krrish-berri-2 rebased onto current main (9cdedf81cd) to clear the conflict. The only conflict was httpxSpecialProvider in litellm/types/llms/custom_http.py, where #43669 and #43885 added ROICalculator and AgentHarness; all three members are kept, and the other two commits are unchanged. Locally: 28/28 in test_nadir_classifier.py, 210/210 across tests/unit/router_strategy/complexity_router/, assert_ci_coverage.py OK, ruff clean.

@tin-berri this plugs in through the existing classifier_plugin hook (decision only, no generation call), so it doesn't touch the opensource_classifier_config providers from #43626 / #44246.

[doramirdor]

Two small follow-ups on top of the rebase:

• 651bd044e3 drops two # mutable-ok comments that chore(lint): remove the LIT002 mutable-construction rule #43971 left behind. With LIT002 gone they suppress nothing, so type_discipline_gate reports them as LIT013.
• df0714e1f7 makes the plugin route on /v1/bucket's routing_tier instead of bucket. routing_tier is the tier Nadir itself acts on, after its confidence and gate adjustments; bucket is the raw prediction. It falls back to bucket only for an older self-hosted server that does not send routing_tier.

Locally: 214/214 across tests/unit/router_strategy/complexity_router/, type_discipline_gate, test_quality_gate and ruff_strict_gate OK against current main, ruff format clean. The previous CI run on 51492b3c62 was cancelled before any job started, so this push is also its first real run.