fixes: ci pipeline for load balancing PR

[Harshit28j]

fixes ci pipeline: #18891

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Feb 3, 2026 2:57pm

[greptile-apps]

Greptile Overview

Greptile Summary

This PR implements access group-based filtering for load balancing to prevent cross-team deployment routing (fixes #18333). However, it accidentally removes critical user spend tracking needed by Prometheus metrics.

Key Changes

• Added get_access_groups_from_models() helper to extract access groups from model lists
• Implemented filter_deployments_by_access_groups() to filter deployments based on team's allowed access groups
• Integrated access group filtering into router's deployment selection logic
• Improved error messages with deployment details and helpful tips for wildcard patterns
• Added null checks for retry_policy and allowed_fails_policy logging

Critical Issue

• Bug: Removed user_api_key_user_spend and user_api_key_user_max_budget from metadata in litellm_pre_call_utils.py:1019-1035. These fields are used by prometheus.py for user budget metrics tracking and must be restored.

Testing

• Comprehensive test coverage for the access group filtering feature

Confidence Score: 1/5

• This PR has a critical bug that breaks user spend tracking for Prometheus metrics
• The PR accidentally removed user spend/budget tracking fields that are actively used by prometheus.py. This will cause user budget metrics to stop working, breaking monitoring functionality. The access group filtering logic itself looks correct and well-tested, but the regression bug is a blocker.
• Pay close attention to litellm/proxy/litellm_pre_call_utils.py - the removed user spend tracking must be restored

Important Files Changed

Filename	Overview
litellm/proxy/litellm_pre_call_utils.py	Accidentally removed user spend/budget metrics needed by prometheus.py, added access group filtering logic
litellm/router_utils/common_utils.py	Added filter_deployments_by_access_groups function to prevent cross-team load balancing, formatting improvements
litellm/router.py	Integrated access group filtering, improved error messages, added null checks for retry/fails policies

Sequence Diagram

sequenceDiagram
    participant User
    participant ProxyServer
    participant PreCallUtils as litellm_pre_call_utils
    participant ModelChecks as model_checks
    participant Router
    participant CommonUtils as router_utils/common_utils
    participant Deployment

    User->>ProxyServer: API Request (team has AG2 access)
    ProxyServer->>PreCallUtils: add_litellm_data_to_request()
    PreCallUtils->>Router: get_model_access_groups()
    Router-->>PreCallUtils: model_access_groups dict
    PreCallUtils->>ModelChecks: get_access_groups_from_models(models)
    ModelChecks-->>PreCallUtils: ["AG2"]
    PreCallUtils->>PreCallUtils: Set metadata["user_api_key_allowed_access_groups"] = ["AG2"]
    PreCallUtils-->>ProxyServer: Request with metadata
    ProxyServer->>Router: async_function_with_fallbacks()
    Router->>Router: get_available_deployment()
    Router->>CommonUtils: filter_deployments_by_access_groups()
    CommonUtils->>CommonUtils: Filter deployments where access_groups overlap with ["AG2"]
    CommonUtils-->>Router: Filtered deployments (only AG2 deployments)
    Router->>Deployment: Route to allowed deployment
    Deployment-->>User: Response

Loading

[greptile-apps]

_{3 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}