Skip to content

fix(proxy): include access group models in /v1/models for teams and keys - #35846

Closed
devin-ai-integration[bot] wants to merge 4 commits into
litellm_internal_stagingfrom
litellm_models_endpoint_access_groups
Closed

devin-ai-integration[bot] wants to merge 4 commits into
litellm_internal_stagingfrom
litellm_models_endpoint_access_groups

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • Access group models were callable but missing from /v1/models
  • Team keys saw only team.models, never the granted access groups
  • /team/info showed the access group, the models endpoint did not

How it solves it:

  • Resolve team and key access_group_ids when listing models
  • Reuses the same helpers request-time auth already falls back to

Relevant issues

Linear ticket

Resolves LIT-5242

https://linear.app/litellm-ai/issue/LIT-5242/v1modelsinclude-model-access-groupstrue-omits-models-granted-via-team

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Same team, same access group, same DB across both runs. Team is restricted to anthropic-haiku-4-5, and a unified access group granting gpt-5.5 and gpt-4o-mini is assigned to it. Real OpenAI calls through the proxy, no mocks

Setup, run once against the proxy

$ TID=$(curl -s localhost:4000/team/new -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
    -d '{"team_alias":"proof","models":["anthropic-haiku-4-5"]}' | jq -r .team_id)

$ curl -s localhost:4000/v1/access_group -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
    -d "{\"access_group_name\":\"proof-ag\",\"access_model_names\":[\"gpt-5.5\",\"gpt-4o-mini\"],\"assigned_team_ids\":[\"$TID\"]}"
{'access_group_id': 'bf2f9fe9-e9fc-4b2d-bbb3-96874004b7e2', 'access_model_names': ['gpt-5.5', 'gpt-4o-mini'], 'assigned_team_ids': ['f6c94972-fb9a-473f-962c-d9f2daf0c7ac']}

$ KV=$(curl -s localhost:4000/key/generate -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
    -d "{\"team_id\":\"$TID\"}" | jq -r .key)

Before, at base commit e2950a89957e459e662105fecb3d2e44056b08b5

$ curl -s "localhost:4000/team/info?team_id=$TID" -H "Authorization: Bearer sk-1234"
{'models': ['anthropic-haiku-4-5'], 'access_group_ids': ['bf2f9fe9-e9fc-4b2d-bbb3-96874004b7e2']}

$ curl -s "localhost:4000/v1/models?include_model_access_groups=true" -H "Authorization: Bearer $KV"
["anthropic-haiku-4-5"]

$ curl -s localhost:4000/chat/completions -H "Authorization: Bearer $KV" -H "Content-Type: application/json" \
    -d '{"model":"gpt-5.5","messages":[{"role":"user","content":"Reply with exactly: access group works"}]}'
{"model": "gpt-5.5", "content": "access group works", "total_tokens": 30, "id": "chatcmpl-E9IrlvkhVduYqaDQ6WXtygAaKITZc"}

gpt-5.5 bills a real request and returns a real completion, so the key can clearly call it, yet the listing hides it. That mismatch is the bug

After, at commit 8390e6d9664f98365b575d100455b1b2e357f0b4, the code as it stands, 2c1b566e12 on top of it only strips comments

$ curl -s "localhost:4000/team/info?team_id=$TID" -H "Authorization: Bearer sk-1234"
{'models': ['anthropic-haiku-4-5'], 'access_group_ids': ['bf2f9fe9-e9fc-4b2d-bbb3-96874004b7e2']}

$ curl -s "localhost:4000/v1/models?include_model_access_groups=true" -H "Authorization: Bearer $KV"
["anthropic-haiku-4-5", "gpt-5.5", "gpt-4o-mini"]

$ curl -s localhost:4000/chat/completions -H "Authorization: Bearer $KV" -H "Content-Type: application/json" \
    -d '{"model":"gpt-5.5","messages":[{"role":"user","content":"Reply with exactly: access group works"}]}'
{"model": "gpt-5.5", "content": "access group works", "total_tokens": 30, "id": "chatcmpl-E9IqyWatd5tnSEWHVW4EWheGD9r4v"}

The listing now matches what the key can call. Nothing new is granted, a model outside both team.models and the access group is still refused

$ curl -s localhost:4000/chat/completions -H "Authorization: Bearer $KV" -H "Content-Type: application/json" \
    -d '{"model":"anthropic-sonnet-4-5","messages":[{"role":"user","content":"hi"}]}'
{"error":{"message":"team not allowed to access model. This team can only access models=['gpt-5.5', 'gpt-4o-mini']. Tried to access anthropic-sonnet-4-5","type":"team_model_access_denied","param":"model","code":"403"}}

Type

🐛 Bug Fix

Changes

get_available_models_for_user built its listing purely from key.models, team.models, and the router level model access groups. It never looked at access_group_ids, the unified LiteLLM_AccessGroupTable assignment the dashboard and /v1/access_group write. Request time auth does look at them: both can_team_access_model and can_key_call_model fall back to those groups when the direct model list rejects a model. So a team could call a model that /v1/models refused to advertise, and callers had to reach for /team/info to discover it

The listing now resolves the same grants through the same helpers rather than duplicating the lookup

all_models = get_complete_model_list(...)          # unchanged

if only_model_access_groups:
    return all_models

access_group_models = await _get_models_from_unified_access_groups(...)
return list(dict.fromkeys(chain(all_models, access_group_models)))

_get_models_from_unified_access_groups mirrors auth exactly: team assigned groups go through auth_checks._get_models_from_access_groups, key assigned groups go through get_authorized_resources_from_key_access_groups, which keeps the ownership check so a key cannot surface a group it was never authorized for. The team object already fetched for an explicit ?team_id= is reused, otherwise the caller's own team is looked up through the existing cache backed get_team_object, and a failed lookup degrades to listing nothing extra instead of erroring the endpoint

only_model_access_groups=true is untouched, it still returns access group names rather than their members. access_group_ids on _get_resources_from_access_groups and _get_models_from_access_groups widened from list[str] to Sequence[str] so a tuple can be passed without a defensive copy

Regression coverage lives in tests/test_litellm/proxy/utils/helpers/test_model_access.py, stubbing only the access group DB boundary. Three of the five new tests fail on base and pass here, the other two pin the behavior that must not drift, a team with no access groups listing exactly its own models and only_model_access_groups not expanding members

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/5be9aaa48f104a4ba378283d70e26290

Models granted to a team or key through a unified access group were callable
but absent from the model listing, because get_available_models_for_user only
read team.models / key.models while request-time auth falls back to
access_group_ids.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@greptile-apps

greptile-apps Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR makes /v1/models include models granted through unified team and key access groups while preserving the access-group-only listing mode.

  • Resolves team access-group IDs through the cached team helper.
  • Reuses existing authorization helpers for team and key group models.
  • Deduplicates resolved models and adds regression coverage for team, key, deduplication, and unchanged behavior.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
litellm/proxy/auth/auth_checks.py Broadens access-group helper inputs from lists to sequences without changing authorization behavior.
litellm/proxy/utils.py Extends model discovery with authorized team and key access-group models while preserving existing special modes.
tests/test_litellm/proxy/utils/helpers/test_model_access.py Adds isolated regression tests covering access-group expansion, deduplication, and unchanged fallback behavior.

Reviews (2): Last reviewed commit: "style(proxy): drop added comments and do..." | Re-trigger Greptile

Comment thread litellm/proxy/utils.py Outdated
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptileai

@codecov

codecov Bot commented Aug 4, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 82.14286% with 5 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
litellm/proxy/utils.py 81.48% 5 Missing ⚠️

📢 Thoughts on this report? Let us know!

@codspeed

codspeed Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing litellm_models_endpoint_access_groups (b92dc44) with litellm_internal_staging (2792887)1

Open in CodSpeed

Footnotes

  1. No successful run was found on litellm_internal_staging (32deaff) during the generation of this report, so 2792887 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

milan-berri and others added 2 commits August 5, 2026 19:51
…access_groups

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…access_groups

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@ryan-crabbe-berri

Copy link
Copy Markdown
Contributor

Closing in favor of #36230, which merged the same access group resolution into get_available_models_for_user and fully covers LIT-5242

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants