fix(router): tag-aware pre-routing strategy selection for shared model_name

[devin-ai-integration]

Relevant issues

Fixes #33655

Linear ticket

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Type

🐛 Bug Fix

Changes

The pre-routing strategy registries (auto_routers, complexity_routers, adaptive_routers, quality_routers) were keyed by model_name alone with a scalar value:

self.complexity_routers: Dict[str, ComplexityRouter] = {}

So two deployments sharing a public model_name but carrying different tags collapsed to one config: registration either kept the first router or raised "... already exists ..." at startup. Because complexity/auto classification runs in async_pre_routing_hook before ordinary deployment tag filtering, a request carrying the second tag was classified/routed through the first deployment's config and then rejected with a 401 (not allowed to access model due to tags configuration) once tag filtering ran.

This mirrors the same handling the issue asks for across all four pre-routing families, not just complexity routers.

Each registry now holds a list of tag-scoped strategies:

self.complexity_routers: dict[str, list[TaggedPreRoutingStrategy["ComplexityRouter"]]] = {}

• TaggedPreRoutingStrategy (new frozen dataclass in types/router.py) pairs a strategy with the tags it was registered under; PreRoutingStrategy (new protocol) is the shared structural type of the four routers.
• _register_pre_routing_strategy centralizes registration for all four families: a repeated (model_name, tags) pair is still rejected, but distinct tags under one model_name now coexist.
• _select_pre_routing_strategy runs in async_pre_routing_hook before classification and picks the entry whose tags match the request tags (via the existing is_valid_deployment_tag / tag_filtering_match_any semantics), then a default-tagged entry, then the first registered — consistent with normal tag-based routing, which only filters later.
• Adaptive-router consumers in proxy/proxy_server.py (startup state load, /adaptive_router/state snapshots, background flusher) iterate the tagged lists.

Selection sketch:

candidates = [*auto_routers.get(model, []), *complexity_routers.get(model, []),
              *adaptive_routers.get(model, []), *quality_routers.get(model, [])]
if len(candidates) <= 1: return candidates[0].strategy if candidates else None
for c in candidates:                     # request tags win
    if c.tags and is_valid_deployment_tag(list(c.tags), request_tags, match_any): return c.strategy
for c in candidates:                     # then a `default`-tagged entry
    if "default" in c.tags: return c.strategy
return candidates[0].strategy            # else first registered

Tests: TestComplexityRouterTagBasedRouting registers two smart complexity deployments tagged cn/us and asserts both register, that each request tag resolves to its own tier model through async_pre_routing_hook, and that same-(model_name, tags) is still rejected. Existing adaptive/proxy tests updated for the list-valued registry shape. These fail against the old scalar registry (which raised at startup or routed both tags through the first config).

Note: this is the broader-scope alternative to #33660 (which fixes complexity routers only); consolidating per maintainer direction.

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/967b3f08af4d47cab5c37f4320a5eac8
Requested by: @krrish-berri-2

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codecov]

Codecov Report

❌ Patch coverage is 89.04110% with 8 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/proxy/proxy_server.py	50.00%	7 Missing ⚠️
litellm/router.py	98.03%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

This PR fixes a bug where multiple deployments sharing a model_name but carrying different tags would collapse to a single pre-routing strategy, causing the second tag to be rejected at startup or routed through the wrong config. The fix applies uniformly across all four pre-routing families (auto, complexity, adaptive, quality).

• Each registry (auto_routers, complexity_routers, adaptive_routers, quality_routers) is changed from Dict[str, RouterType] to Dict[str, list[TaggedPreRoutingStrategy[RouterType]]], keyed by (model_name, tags) pairs via the new TaggedPreRoutingStrategy frozen dataclass.
• _select_pre_routing_strategy replaces the former .get(model) chain; it resolves the right strategy by matching request tags against each entry's tags field, falling back to a "default"-tagged entry and then the first-registered.
• All consumer sites in proxy_server.py (startup state load, flusher loop, /adaptive_router/state endpoint) are updated to iterate the new list shape, and the existing test fixtures are updated accordingly.

Confidence Score: 4/5

Safe to merge with minor concerns about the unmatched-tag fallback path and a missing test for it.

The registry refactoring and selection logic are mechanically sound — registration, deduplication, and tag-matching all behave correctly. The one behavioral edge worth watching is that a request whose tags match no registered strategy silently falls through to the first-registered entry instead of returning None, which can route requests through a mismatched regional config when tag filtering is not enforced at the deployment level. A test covering this fallback path is also absent. These are non-blocking in practice because the post-routing tag filter will catch the mismatch when enable_tag_filtering=True.

litellm/router.py (_select_pre_routing_strategy) and tests/test_litellm/router_strategy/test_complexity_router.py (missing fallback test case).

Important Files Changed

Filename	Overview
litellm/router.py	Core change: adds _select_pre_routing_strategy, _register_pre_routing_strategy, _has_registered_strategy, and _deployment_tags helpers; all four strategy registries now hold list[TaggedPreRoutingStrategy[...]] instead of a scalar value. Selection logic is generally correct but has a subtle case where multiple candidates exist (both a complexity router and its promoted adaptive router share the same tags) — the complexity router always wins due to list ordering, which is the intended behavior.
litellm/types/router.py	Adds TaggedPreRoutingStrategy (frozen dataclass, covariant TypeVar, slots=True) and PreRoutingStrategy Protocol (runtime_checkable). Both types are well-constructed; the covariant TypeVar is appropriate for the read-only frozen dataclass.
litellm/proxy/proxy_server.py	Three sites updated to iterate through the new tagged list shape: startup state load, adaptive flusher loop, and /adaptive_router/state snapshot endpoint. All three transformations are mechanically correct and preserve prior behavior.
tests/test_litellm/router_strategy/test_complexity_router.py	New TestComplexityRouterTagBasedRouting class tests the two-strategy registration, the duplicate-tag rejection, and the happy-path tag-to-strategy dispatch. Uses local heuristic classification (no network calls). Missing a test case for the no-match fallback (when request tags don't match any strategy's tags).
tests/router_unit_tests/test_router_helper_utils.py	Two existing tests updated to reflect new list-valued registry: success assertion uses [0].strategy, duplicate test pre-seeds with TaggedPreRoutingStrategy and updates error message regex. Coverage preserved.
tests/test_litellm/router_strategy/adaptive_router/test_router_dispatch.py	Accessor pattern updated via new _adaptive(r, name) helper that extracts [0].strategy. All existing test assertions are preserved and the helper correctly reflects the new single-tag-entry shape.
tests/test_litellm/router_strategy/adaptive_router/test_state_endpoint.py	Added _entry() helper to wrap _make_router() in TaggedPreRoutingStrategy; all three test scenarios that mock fake_router.adaptive_routers are correctly updated.
tests/test_litellm/proxy/proxy_server/test_background_health.py	Mock updated to wrap fake_ar in a single-element TaggedPreRoutingStrategy list. Flush-and-load logic is still exercised correctly.
tests/test_litellm/proxy/proxy_server/test_routes_misc.py	Mock updated to wrap the bandit in a TaggedPreRoutingStrategy list. Snapshot endpoint behavior is unchanged.

Comments Outside Diff (1)

1. tests/test_litellm/router_strategy/test_complexity_router.py, line 706-721 (link)

   No test for the no-match fallback path

   TestComplexityRouterTagBasedRouting covers both happy paths ("cn" → "gpt-cn", "us" → "gpt-us") and the duplicate-tag rejection, but doesn't test what happens when the request carries a tag that matches neither registered strategy (e.g. tags=["eu"]). Per _select_pre_routing_strategy, the fallback is candidates[0].strategy (first registered). Adding a test for that case would lock in the documented fallback behavior and prevent accidental regressions if the selection logic changes.

   Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

_{Reviews (1): Last reviewed commit: "test(router): cover tag-scoped pre-routi..." | Re-trigger Greptile}

[greptile-apps]

Unmatched tags fall through to first-registered strategy

When request_tags is non-empty but doesn't match any candidate's tags, the loop exits silently and the code falls through to the "default" check and then candidates[0].strategy. A request tagged ["eu"] against only ["cn"]/["us"] strategies will pre-route through the cn config, then be rejected at the post-routing tag filter — correct overall, but the pre-routing step is doing unnecessary work with a mismatched config. If enable_tag_filtering is off this also means the eu request is silently forwarded through the wrong regional tier without any error. Adding an explicit return None (or an explicit log) when request tags are present but match nothing would make the fallback intentional rather than implicit.

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_tag_aware_pre_routing_strategies (a87adb8) with litellm_internal_staging (4d33964)}

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 2 additional findings.