Repository navigation
fix(proxy): list key and team model aliases in GET /v1/models - #42908
Conversation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
|
|
|
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
bugbot run |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
| user_api_key_dict.aliases, user_api_key_dict.team_model_aliases, user_api_key_dict.team_id, team_id | ||
| ), | ||
| ) | ||
| or model_id, |
There was a problem hiding this comment.
Alias retrieve overwrites response id
Medium Severity
GET /v1/models/{alias} resolves the alias through resolve_public_name and then sets id from internal_to_public. When the target is a team-scoped model, that map replaces the requested alias with the team's public name, so retrieve returns a different id than the listing row for the same alias.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit d6ec911. Configure here.
There was a problem hiding this comment.
Fixed in f187b8a: when the request is an alias the response id is the alias, regression test added for a team scoped target
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…p the alias as the retrieved id Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
bugbot run |
…ly malformed alias entries Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
bugbot run |
| resolved_model_id: Final = TeamModelNameTranslator.resolve_public_name( | ||
| model_id=model_id, | ||
| model_id=aliased_model_id or model_id, | ||
| available_models=all_models, |
There was a problem hiding this comment.
Listed ID retrieves different model
If an undiscoverable team model and a discoverable model share a public ID, /v1/models filters out the undiscoverable model and shows the discoverable model’s metadata. This retrieval path instead resolves the ID using all_models, which still contains the undiscoverable model. If that model comes first, GET /v1/models/{id} returns its metadata for the ID shown in the list, so clients see conflicting information for the same model.
There was a problem hiding this comment.
Pre-existing and out of scope: model_info has always resolved through all_models including undiscoverable deployments, and test_v1_models_by_id_still_serves_the_hidden_model_to_non_admin pins that retrieval. This PR only excludes undiscoverable ids from the alias guard so alias resolution mirrors listing; a public id shared by a hidden and a visible deployment is a config collision the existing first-occurrence resolver already handled this way before this change.
There was a problem hiding this comment.
You're right — the existing model_info contract intentionally resolves direct IDs through all_models, including undiscoverable deployments, and test_v1_models_by_id_still_serves_the_hidden_model_to_non_admin pins that behavior. This PR only removes undiscoverable IDs from the alias collision guard so alias resolution matches the list; it does not introduce the hidden/visible public-ID collision behavior. I’m withdrawing this comment.
…alias guard Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
bugbot run |
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
bugbot run |
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 2557744. Configure here.
* fix(proxy): list key and team model aliases in GET /v1/models
* fix(proxy): keep alias listing helpers within the type discipline budget
* test(proxy): cover alias rows on GET /v1/models and /v1/models/{id}
* fix(proxy): apply team then key aliases like chat completions and keep the alias as the retrieved id
* fix(proxy): apply key aliases twice like chat completions and skip only malformed alias entries
* fix(proxy): apply the global model_alias_map between the key alias passes like chat completions
* fix(proxy): list only the caller's own aliases and never rewrite a listed model id on retrieval
* style(proxy): ruff format model_info alias lookup
* fix(proxy): hide undiscoverable names from model retrieval so an alias named like one resolves to its target
* fix(proxy): keep undiscoverable models retrievable by id while excluding them from the alias guard
* refactor(proxy): pass an immutable name sequence into the model_info alias guard
* test(proxy): type the model list alias test helpers
* test(proxy): annotate the new alias listing test fixtures and helpers
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>


TLDR
Problem this solves:
model_aliasesroute on/chat/completionsbut never show inGET /v1/modelsHow it solves it:
/v1/modelsadds one row per caller alias whose target the caller can already listmodel_alias_map, then key again, the passes/chat/completionsruns, so list, retrieval and routing agree on one target/v1/models/{alias}returns the target's metadata under the requested alias id, also when the target is a team scoped modelUser Flow
Before: a Claude Code user on a team whose admin aliased
claude-sonnet-4-5togpt-4.1-minicannot see that alias in the model list, even though it works"model": "claude-sonnet-4-5"and a team JWT, and get a 200 answered bygpt-4.1-minianthropic-version, and get["gpt-4.1-mini"]onlyclaude-sonnet-4-5entry, and GET https://litellm-domain/v1/models/claude-sonnet-4-5 returns 400does not exist or is not accessibleAfter: the same user sees the alias in the list and can pick it
"model": "claude-sonnet-4-5"and a team JWT, and get a 200 answered bygpt-4.1-minianthropic-version, and get["gpt-4.1-mini", "claude-sonnet-4-5"]claude-sonnet-4-5[1m], and GET https://litellm-domain/v1/models/claude-sonnet-4-5 returns 200 with the target's metadataRelevant issues
Fixes #37034
Linear ticket
Resolves LIT-8515
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Setup shared by both legs: live proxy on localhost:4000 with Postgres, JWT auth against a local JWKS, a team
repro-team-9069withmodels: [gpt-4.1-mini]andmodel_aliases: {claude-sonnet-4-5: gpt-4.1-mini}, a JWT whosegroupsclaim binds the caller to that team, and real OpenAI calls.$JWTis that token. Screen recording of the run below is linked from the internal Slack thread: https://berriaillm.slack.com/archives/C0C04J86WTC/p1790237318325969?thread_ts=1790235554.889039&cid=C0C04J86WTCBefore (09ebb28)
Alias routes on chat completions
curl -s localhost:4000/v1/chat/completions -H "Authorization: Bearer $JWT" -H 'content-type: application/json' -d '{"model":"claude-sonnet-4-5","messages":[{"role":"user","content":"say pong"}],"max_tokens":5}'model: claude-sonnet-4-5 | content: Pong!OpenAI shape list
curl -s localhost:4000/v1/models -H "Authorization: Bearer $JWT"["gpt-4.1-mini"]Anthropic shape list
curl -s localhost:4000/v1/models -H "Authorization: Bearer $JWT" -H "anthropic-version: 2023-06-01"["gpt-4.1-mini"]Claude Code picker list
curl -s localhost:4000/v1/models -H "Authorization: Bearer $JWT" -H "anthropic-version: 2023-06-01" -H "user-agent: claude-cli/2.1.267 (external, cli)"["claude-router-6770742d342e312d6d696e69[1m]"]Alias model info
curl -s localhost:4000/v1/models/claude-sonnet-4-5 -H "Authorization: Bearer $JWT"{"detail":"The modelclaude-sonnet-4-5does not exist or is not accessible"}After (2eb9a1d, whose behavior 579abf4, 4a06c64, 459bbd3, 8e970ce, 884653a and 2557744 keep for this flow; identical output on a3a2a1a, d6ec911, f187b8a, 8a205fb and 470725b)
Alias routes on chat completions
curl -s localhost:4000/v1/chat/completions -H "Authorization: Bearer $JWT" -H 'content-type: application/json' -d '{"model":"claude-sonnet-4-5","messages":[{"role":"user","content":"say pong"}],"max_tokens":5}'model: claude-sonnet-4-5 | content: Pong!OpenAI shape list
curl -s localhost:4000/v1/models -H "Authorization: Bearer $JWT"["gpt-4.1-mini", "claude-sonnet-4-5"]Anthropic shape list
curl -s localhost:4000/v1/models -H "Authorization: Bearer $JWT" -H "anthropic-version: 2023-06-01"["gpt-4.1-mini", "claude-sonnet-4-5"]Claude Code picker list
curl -s localhost:4000/v1/models -H "Authorization: Bearer $JWT" -H "anthropic-version: 2023-06-01" -H "user-agent: claude-cli/2.1.267 (external, cli)"["claude-router-6770742d342e312d6d696e69[1m]", "claude-sonnet-4-5[1m]"]Alias model info
curl -s localhost:4000/v1/models/claude-sonnet-4-5 -H "Authorization: Bearer $JWT"{"id":"claude-sonnet-4-5","object":"model","created":1677610602,"owned_by":"openai","mode":"chat","max_input_tokens":1047576,"max_output_tokens":32768}Master key on the After proxy still lists
["gpt-4.1-mini", "gpt-4.1"], so team aliases stay scoped to the callerType
🐛 Bug Fix
Caveats (if any)
Medium
GET /v1/models/{id}, the alias is not shown twice, while/chat/completionsstill rewrites that name through the aliasGET /v1/models/{id}uses the same undiscoverable filter as the list, so an alias named like an undiscoverable model resolves to the alias target, while undiscoverable models themselves stay retrievable by idclaude-cliuser agent and a unit test at that user agentLow
owned_byand token limits are the target'smodel_alias_mapnames are a rewrite hop only, they do not become rows in the caller's listreturn_wildcard_routes=truedoes not expand aliases against wildcard patternsQA runbook
Needs the e2e stack from tests/e2e/CONTRIBUTING.md: proxy with JWT auth against the local Keycloak,
OPENAI_API_KEYfor the cheap OpenAI model, master key for team creation{"team_id": "<group>", "team_alias": "aliased", "models": ["gpt-4.1-mini"], "model_aliases": {"my-alias": "gpt-4.1-mini"}}{"model": "my-alias", ...}, expect 200 with a non-empty choices arraydata[].idto contain bothmy-aliasandgpt-4.1-minianthropic-version: 2023-06-01, expectdata[].idto contain bothmy-aliasandgpt-4.1-miniFinal Attestation
Link to Devin session: https://app.devin.ai/sessions/819b5689891f44fb88c984dd8c4149fb
Open in Devin Desktop: https://app.devin.ai/desktop/session/819b5689891f44fb88c984dd8c4149fb?variant=devin
Note
Medium Risk
Changes model discovery and
/v1/models/{id}resolution for authenticated callers; behavior is scoped to alias presentation but must stay aligned with existing routing and team scoping.Overview
GET
/v1/modelsand/v1/models/{id}now expose key and teammodel_aliasesthe same way/chat/completionsalready routes them, so managed clients (e.g. Claude Code under JWT) can discover alias names next to their targets in OpenAI and Anthropic list shapes.New listing helpers in
model_listing_utilsbuild caller-specific alias maps (team maps only when listing the authenticated team), append one list row per alias whose fully rewritten target is already listable, and resolve alias lookups using the same pass order as completions: team → key → globalmodel_alias_map→ key again. Collisions with an existing listed id keep the original row; malformed map entries are skipped; global alias names rewrite but are not added as rows.model_infoapplies the same rewrite for{id}lookups, returns metadata under the requested alias id, and aligns undiscoverable-model handling with the list so alias names like an undiscoverable deployment still resolve to the alias target.Reviewed by Cursor Bugbot for commit 2557744. Bugbot is set up for automated code reviews on this repo. Configure here.