fix: enforce project-level model-specific rate limits in parallel_req…

[shivamrawat1]

Root Cause

Project-level model-specific rate limits (model_rpm_limit / model_tpm_limit stored
in project_metadata) were never checked during rate limit enforcement. The
parallel_request_limiter_v3.py had descriptor creation for team and organization
model limits, but nothing for projects.

Changes Made

litellm/proxy/auth/auth_utils.py

• Extended get_model_rate_limit_from_metadata's Literal type to also accept
  "project_metadata"
• Added get_project_model_rpm_limit() and get_project_model_tpm_limit() helpers
  (mirrors the existing team equivalents)

litellm/proxy/hooks/parallel_request_limiter_v3.py

• Added _add_project_model_rate_limit_descriptor_from_metadata() method — mirrors
  _add_team_model_rate_limit_descriptor_from_metadata but reads from project_metadata
  and creates a "model_per_project" descriptor keyed as "{project_id}:{model}"
• Called it in async_pre_call_hook after the team metadata step

tests/test_litellm/proxy/hooks/test_parallel_request_limiter_v3.py

• Added test_project_model_rate_limits_enforced_v3 — verifies the model_per_project
  descriptor is created with correct rpm/tpm values
• Added test_project_model_rate_limits_not_triggered_for_other_model_v3 — verifies
  no descriptor is created when the requested model isn't in the project's limit
  config

Project level rpm 1

Before:

After:

[greptile-apps]

Greptile Summary

This PR adds enforcement of project-level model-specific rate limits (model_rpm_limit / model_tpm_limit from project_metadata) in parallel_request_limiter_v3.py. The pre-call hook now creates a model_per_project descriptor keyed as {project_id}:{model}, and the success event handler (_build_success_event_pipeline_operations) correctly increments the model_per_project counter with actual token usage, mirroring the existing model_per_team and model_per_organization patterns.

Confidence Score: 5/5

Safe to merge — the pre-call descriptor and post-call TPM increment for model_per_project are both correctly implemented, mirroring the established team/org patterns.

The previously flagged P1 concern (missing model_per_project TPM increment in the success handler) has been addressed in this PR at lines 1688–1696. All remaining notes (dead helper functions, redundant metadata lookups, project_id=None bucket) are in the previous threads and are P2-level at most, none blocking merge.

No files require special attention; all critical paths are covered by the new tests.

Important Files Changed

Filename	Overview
litellm/proxy/hooks/parallel_request_limiter_v3.py	Adds _add_project_model_rate_limit_descriptor_from_metadata for pre-call enforcement and adds model_per_project TPM increment in the success event pipeline; mirrors team/org patterns correctly.
litellm/proxy/auth/auth_utils.py	Extends get_model_rate_limit_from_metadata Literal type for project_metadata and adds get_project_model_rpm_limit / get_project_model_tpm_limit helpers (currently unused in the new method, which calls get_model_rate_limit_from_metadata directly).
tests/test_litellm/proxy/hooks/test_parallel_request_limiter_v3.py	Adds two regression tests for project-level model rate limits: one verifying descriptor creation, one verifying no descriptor for an unlisted model. Both are mock-only with no network calls.
tests/test_litellm/proxy/auth/test_auth_utils.py	Adds unit tests for get_project_model_rpm_limit, get_project_model_tpm_limit, and check_complete_credentials helpers — all mock-based with good coverage.

Sequence Diagram

sequenceDiagram
    participant Client
    participant PreCallHook as async_pre_call_hook
    participant Descriptor as _add_project_model_rate_limit_descriptor
    participant RateLimiter as should_rate_limit
    participant SuccessHook as _build_success_event_pipeline_operations
    participant Redis

    Client->>PreCallHook: request with project_metadata model limits
    PreCallHook->>Descriptor: build descriptors
    Descriptor->>Descriptor: get_model_rate_limit_from_metadata rpm
    Descriptor->>Descriptor: get_model_rate_limit_from_metadata tpm
    Descriptor-->>PreCallHook: RateLimitDescriptor model_per_project
    PreCallHook->>RateLimiter: check all descriptors
    RateLimiter-->>PreCallHook: OK or 429
    PreCallHook-->>Client: proceed or rate limited

    Client->>SuccessHook: on success with token usage
    SuccessHook->>Redis: increment model_per_project tokens counter

Loading

_{Reviews (7): Last reviewed commit: "Merge remote-tracking branch 'upstream/l..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 87.87879% with 8 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/proxy/hooks/parallel_request_limiter_v3.py	86.20%	8 Missing ⚠️

📢 Thoughts on this report? Let us know!

[gitguardian]

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secret in your pull request

GitGuardian id	GitGuardian status	Secret	Commit	Filename
29203065	Triggered	JSON Web Token	a449fc1	tests/test_litellm/proxy/test_litellm_pre_call_utils.py	View secret

🛠 Guidelines to remediate hardcoded secrets

1. Understand the implications of revoking this secret by investigating where it is used in your code.
2. Replace and store your secret safely. Learn here the best practices.
3. Revoke and rotate this secret.
4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

• following these best practices for managing and storing secrets including API keys and other credentials
• install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

---

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}

[shivamrawat1]

@greptile review again

[shivamrawat1]

@greptile review with the new commit