Skip to content

fix(proxy): load team member RPM/TPM from membership budget in combined_view - #24925

Merged
krrish-berri-2 merged 1 commit into
BerriAI:litellm_oss_staging_04_01_2026from
milan-berri:fix/team-member-rpm-tpm-combined-view
Apr 2, 2026
Merged

fix(proxy): load team member RPM/TPM from membership budget in combined_view#24925
krrish-berri-2 merged 1 commit into
BerriAI:litellm_oss_staging_04_01_2026from
milan-berri:fix/team-member-rpm-tpm-combined-view

Conversation

@milan-berri

@milan-berri milan-berri commented Apr 1, 2026

Copy link
Copy Markdown
Contributor

Relevant issues

N/A — fixes virtual keys not receiving per-team-member RPM/TPM from combined_view when those limits are stored on the team membership budget row (LiteLLM_TeamMembership.budget_idLiteLLM_BudgetTable).

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

  • 50-55 passing tests: main is stable with minor issues.
  • 45-49 passing tests: acceptable but needs attention
  • <= 40 passing tests: unstable; be careful with your merges and assess the risk.
  • Branch creation CI run
    Link:

  • CI run for the last commit
    Link:

  • Merge / cherry-pick CI run
    Links:

Type

🐛 Bug Fix

✅ Test

Changes

Problem: The proxy combined_view query joined LiteLLM_TeamMembership for spend but did not join the membership budget table. Per-team-member RPM/TPM configured on the membership budget were not selected, so team_member_rpm_limit / team_member_tpm_limit were not populated on virtual key auth and parallel_request_limiter_v3 did not enforce those limits.

Fix: Add LEFT JOIN "LiteLLM_BudgetTable" AS b_tm ON tm.budget_id = b_tm.budget_id and select b_tm.tpm_limit AS team_member_tpm_limit, b_tm.rpm_limit AS team_member_rpm_limit in litellm/proxy/utils.py (PrismaClient combined view SQL).

Tests: Add test_team_member_rate_limits_v3_raises_429_when_over_limit in tests/test_litellm/proxy/hooks/test_parallel_request_limiter_v3.py, matching the existing pattern for key RPM limits (mock should_rate_limitOVER_LIMIT for team_member descriptor → assert HTTP 429 and headers).

Join LiteLLM_BudgetTable as b_tm on team membership budget_id and select
team_member_tpm_limit / team_member_rpm_limit so virtual key auth populates
limits for parallel_request_limiter_v3.

Add test_team_member_rate_limits_v3_raises_429_when_over_limit mirroring
existing key-level OVER_LIMIT / HTTP 429 coverage.

Made-with: Cursor
@vercel

vercel Bot commented Apr 1, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Ready Ready Preview, Comment Apr 1, 2026 2:27pm

Request Review

@codspeed-hq

codspeed-hq Bot commented Apr 1, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 16 untouched benchmarks


Comparing milan-berri:fix/team-member-rpm-tpm-combined-view (d668d4c) with main (33c3f13)

Open in CodSpeed

@greptile-apps

greptile-apps Bot commented Apr 1, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes a missing LEFT JOIN in the combined_view SQL query used by PrismaClient.get_data() so that per-team-member RPM/TPM limits stored on LiteLLM_TeamMembership.budget_idLiteLLM_BudgetTable are now correctly surfaced as team_member_rpm_limit / team_member_tpm_limit on virtual key auth, enabling parallel_request_limiter_v3 to enforce those limits.

Changes:

  • litellm/proxy/utils.py: Adds LEFT JOIN \"LiteLLM_BudgetTable\" AS b_tm ON tm.budget_id = b_tm.budget_id and selects b_tm.tpm_limit AS team_member_tpm_limit / b_tm.rpm_limit AS team_member_rpm_limit. The LiteLLM_VerificationTokenView model (which UserAPIKeyAuth inherits) already had these fields typed — the missing piece was just the SQL JOIN to populate them.
  • tests/test_litellm/proxy/hooks/test_parallel_request_limiter_v3.py: Adds a mock test verifying that when should_rate_limit returns OVER_LIMIT for the team_member descriptor, the pre-call hook raises HTTP 429 with the correct rate_limit_type and retry-after headers.
  • The fix is minimal, surgical, and backwards-compatible — the LEFT JOIN returns NULL when no membership budget exists, which falls back to the existing None default in the model.

Confidence Score: 5/5

Safe to merge — the SQL change is a correct additive LEFT JOIN on an already-defined alias, and the test is a well-structured mock unit test following existing patterns.

Both changed files are clean: the SQL JOIN is logically correct (LEFT JOIN on an FK, NULLs handled gracefully), the model fields were already defined, the cast path was already wired up, and the new test follows the established mock-only pattern for this directory. No backward-incompatible changes, no security concerns, no missing migration needed (query-only change).

No files require special attention.

Important Files Changed

Filename Overview
litellm/proxy/utils.py Adds LEFT JOIN on LiteLLM_BudgetTable aliased as b_tm to populate team_member_tpm_limit and team_member_rpm_limit from the membership budget row in the combined_view SQL query
tests/test_litellm/proxy/hooks/test_parallel_request_limiter_v3.py Adds a mock-only test verifying that when should_rate_limit returns OVER_LIMIT for the team_member descriptor, async_pre_call_hook raises HTTP 429 with correct rate_limit_type and retry-after headers

Reviews (1): Last reviewed commit: "fix(proxy): include team membership budg..." | Re-trigger Greptile

@krrish-berri-2
krrish-berri-2 changed the base branch from main to litellm_oss_staging_04_01_2026 April 2, 2026 03:10
@krrish-berri-2
krrish-berri-2 merged commit d1fb63a into BerriAI:litellm_oss_staging_04_01_2026 Apr 2, 2026
55 of 60 checks passed
Sameerlite pushed a commit that referenced this pull request Apr 16, 2026
…PM (#24925)

Join LiteLLM_BudgetTable as b_tm on team membership budget_id and select
team_member_tpm_limit / team_member_rpm_limit so virtual key auth populates
limits for parallel_request_limiter_v3.

Add test_team_member_rate_limits_v3_raises_429_when_over_limit mirroring
existing key-level OVER_LIMIT / HTTP 429 coverage.

Made-with: Cursor
Sameerlite pushed a commit that referenced this pull request May 1, 2026
…PM (#24925)

Join LiteLLM_BudgetTable as b_tm on team membership budget_id and select
team_member_tpm_limit / team_member_rpm_limit so virtual key auth populates
limits for parallel_request_limiter_v3.

Add test_team_member_rate_limits_v3_raises_429_when_over_limit mirroring
existing key-level OVER_LIMIT / HTTP 429 coverage.

Made-with: Cursor
yugborana pushed a commit to yugborana/litellm that referenced this pull request Jun 2, 2026
…PM (BerriAI#24925)

Join LiteLLM_BudgetTable as b_tm on team membership budget_id and select
team_member_tpm_limit / team_member_rpm_limit so virtual key auth populates
limits for parallel_request_limiter_v3.

Add test_team_member_rate_limits_v3_raises_429_when_over_limit mirroring
existing key-level OVER_LIMIT / HTTP 429 coverage.

Made-with: Cursor
fzowl pushed a commit to fzowl/litellm that referenced this pull request Jun 24, 2026
…PM (BerriAI#24925)

Join LiteLLM_BudgetTable as b_tm on team membership budget_id and select
team_member_tpm_limit / team_member_rpm_limit so virtual key auth populates
limits for parallel_request_limiter_v3.

Add test_team_member_rate_limits_v3_raises_429_when_over_limit mirroring
existing key-level OVER_LIMIT / HTTP 429 coverage.

Made-with: Cursor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants