fix(router): resolve model_group_alias in pre-routing hook - #27540
fix(router): resolve model_group_alias in pre-routing hook#27540Jwrede wants to merge 3 commits into
Conversation
[Infra] Promote Internal Staging to main
When a model_group_alias points to a complexity_router (or auto_router, quality_router, adaptive_router) group, calling completion() with the alias name skips the pre-routing hook because the alias is not found in the router maps. The hook returns None, the raw auto_router deployment is selected, and litellm.acompletion fails with "Unmapped LLM provider". Resolve aliases at the top of async_pre_routing_hook using the existing _get_model_from_alias() utility so the resolved group name is used for all router map lookups. Fixes BerriAI#27473
Greptile SummaryThis PR fixes
Confidence Score: 4/5Safe to merge; the change is a minimal, well-scoped guard in a pre-routing hook with no impact on non-alias flows. The production fix is four lines touching only No files require special attention;
|
| Filename | Overview |
|---|---|
| litellm/router.py | Adds a 4-line alias resolution guard at the top of async_pre_routing_hook using the existing _get_model_from_alias helper; minimal, targeted, and consistent with the sync path. |
| tests/test_litellm/router_utils/test_pre_routing_hook_alias_resolution.py | New mock-only unit tests covering alias resolution for complexity, auto, and quality routers, plus direct-name and unknown-model cases; adaptive_routers branch is not tested. |
Reviews (1): Last reviewed commit: "fix(router): resolve model_group_alias b..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| router.auto_routers[target_group] = mock_sub_router | ||
| elif router_type == "quality": | ||
| router.quality_routers[target_group] = mock_sub_router | ||
|
|
||
| return router | ||
|
|
There was a problem hiding this comment.
adaptive_routers alias resolution
async_pre_routing_hook has four router-map branches: auto_routers, complexity_routers, adaptive_routers, and quality_routers. The suite covers three of them but skips adaptive_routers. If a regression were introduced that accidentally excluded adaptive_routers from the alias-resolution path, these tests would not catch it.
There was a problem hiding this comment.
All four router types are covered -- see test_alias_resolves_to_adaptive_router in the test file. The _make_router_with_alias helper accepts router_type="adaptive" and the test class has a dedicated test case for it.
Cover the adaptive_routers branch in async_pre_routing_hook, completing test coverage for all four router map types.
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
|
Any update on the PR review for this issue? Would it be possible to get another reviewer to help move this forward? |
|
Friendly ping @ishaan-jaff @krrish-berri-2 -- this fixes model_group_alias resolution in the pre-routing hook (causes 400 errors with complexity_router/auto_router). Happy to address any feedback. |
|
Hi any update on the MR? The issue it's linked to is a blocker. |
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. |
5 participants
Relevant issues
Fixes #27473
Pre-Submission checklist
tests/test_litellm/directory, Adding at least 1 test is a hard requirementmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewType
Bug Fix
Changes
Router.async_pre_routing_hooklooks upmodelinself.complexity_routers,self.auto_routers,self.quality_routers, andself.adaptive_routers-- but when the caller passes amodel_group_aliasname (e.g.claude-sonnet-4-6) instead of the actual group name (e.g.internal-ai), none of the maps match. The hook returnsNone, the rawauto_router/complexity_routerdeployment is selected as-is, andlitellm.acompletion(model="auto_router/complexity_router")fails withBadRequestError: Unmapped LLM provider.The fix adds alias resolution via
self._get_model_from_alias(model)at the top ofasync_pre_routing_hook, before any router map lookups. This is the same utility already used in_common_checks_available_deploymentfor the sync path.Test plan
Added 6 unit tests in
tests/test_litellm/router_utils/test_pre_routing_hook_alias_resolution.py:hiddenkey) resolves correctly