[Fix] Team-based model name corruption on PATCH - #27376
Conversation
For team-scoped ("Team-BYOK") models, the backend stores an internal
routing key `model_name_{team_id}_{uuid}` in the `model_name` column and
the user-facing name in `model_info.team_public_model_name`. The
internal name was leaking into `/v1/model/info` and `/v2/model/info`
responses. The UI bound the model edit form to that field, so any
non-rename edit (e.g. changing TPM) would PATCH the model with the
internal name as `model_name`. The PATCH endpoint then treated it as
a new public name, overwriting `team_public_model_name` and rewriting
the team's `models[]` list with the mangled string.
Translate `model_name` to the public name in API responses for
team-scoped rows; the DB column and router index continue to use the
internal name as the routing key. Harden `_get_public_model_name` to
detect a no-op or internal-shape `model_name` in the patch and fall
through to the existing `team_public_model_name` instead of treating
it as a rename. Standardize `AgentBuilderView` selection on
`model_info.id` so cross-team agents with the same public name don't
collide.
d77f5c6 to
c96d471
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Greptile SummaryThis PR fixes a data-corruption bug where the internal routing key (
Confidence Score: 4/5Safe to merge with minor ordering and name-restriction edge cases worth addressing in a follow-up. The core corruption fix is sound and presentation-only — DB and router state are never touched. Two minor edge cases exist: BYOK rows sort by their internal routing key when sortBy=model_name (sort runs before translation), and the is_internal_shape prefix guard is slightly over-broad, blocking human-chosen names that happen to share the routing-key prefix. The sort-then-translate ordering in proxy_server.py and the over-broad internal-shape check in model_management_endpoints.py are the two spots worth a follow-up look.
|
| Filename | Overview |
|---|---|
| litellm/proxy/management_endpoints/model_management_endpoints.py | Hardens _get_public_model_name to detect and suppress round-tripped internal routing-key names; logic is correct for the primary BYOK fix path. |
| litellm/proxy/proxy_server.py | Adds _translate_model_name_for_response and applies it in _get_proxy_model_info and model_info_v2; the translation fires after sorting, meaning sort-by-model_name returns BYOK rows in routing-key order, not public-name order. |
| ui/litellm-dashboard/src/components/playground/chat_ui/AgentBuilderView.tsx | Switches agent selection key to model_info.id (with model_name fallback), fixes post-create/update/delete selection to use stable id, and returns list from loadAgents to avoid stale-closure issues. |
Reviews (2): Last reviewed commit: "[Fix] Address review feedback on AgentBu..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
- Apply Black formatting to _get_public_model_name (lint fix). - AgentBuilder: introduce getAgentSelectionKey(agent) returning model_info.id with a fallback to model_name. Use it for the sidebar onClick, find/some lookups, and the active-state comparison so config-file-defined agents (which have no DB id) remain selectable. - AgentBuilder handleSaveAgent: use the create response's model_id to select the just-created agent rather than a model_name lookup, since two team-scoped agents can legitimately share the same public name.
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. |
1 participant
Relevant issues
Summary
Failure Path (Before Fix)
For team-scoped ("Team-BYOK") models, the backend stores two names:
model_name(DB column): an internal routing keymodel_name_{team_id}_{uuid}, used by the in-memory router's primarymodel_name → deploymentindex. Two teams with the same public name need distinct routing keys here.model_info.team_public_model_name: the user-facing public name (e.g.gpt-4o).The internal name was leaking into
/v1/model/infoand/v2/model/infoAPI responses. The UI bound the model edit form to that field, so any non-rename edit (e.g. changing TPM) PATCHed the model with the internal name asmodel_name. The PATCH endpoint then treated it as a new public name, overwroteteam_public_model_name, and rewrote the team'smodels[]list with the mangled string — breaking team-based routing for that model.Fix
Add
_translate_model_name_for_responseinproxy_server.py. For team-scoped rows, swapmodel_nameto the public name held inmodel_info.team_public_model_namebefore returning. Apply at_get_proxy_model_info(covers/v1/model/info,/model/info) and in/v2/model/infobefore pagination. The DB column and router index keep the internal name as the routing key — the swap is presentation-layer only.Harden
_get_public_model_nameinmodel_management_endpoints.py. Whenpatch_data.model_namematches the internal mangled shapemodel_name_{team_id}_{uuid}or equals the existing DB column, treat it as "no rename intent" and fall through to the existingteam_public_model_name. This protects against any client (UI, scripts, future code) that round-trips the leaked internal name.Standardize
AgentBuilderViewselection onmodel_info.idinstead ofmodel_name. Once response translation lands, two team-scoped agents can share a public name; the lookup needs a stable id to avoid collisions.Testing
Verified end-to-end against a local proxy with a regression harness exercising three PATCH scenarios on fixtures including a deliberate cross-team public-name collision:
team_public_model_nameand the team'smodels[]list are unchanged after the PATCH; onlylitellm_params.tpmupdates.models[]list updates correctly.model_name_{team_id}_{uuid}payload — treated as a no-op for the public name; no corruption.Across all scenarios, the DB
model_namecolumn for team rows is unchanged (router invariant preserved). Two fresh captures on the fixed backend produce zero diffs against each other (determinism).Type
🐛 Bug Fix
Screenshots