fix(test): add missing Prometheus metric labels to test_proxy_failure_metrics

[shin-bot-litellm]

Problem

test_proxy_failure_metrics in tests/otel_tests/test_prometheus.py was failing in CI:

https://app.circleci.com/pipelines/github/BerriAI/litellm/56743/workflows/9f492c05-c440-4c80-84d4-34c6cd566930/jobs/1140648/tests

The test expects specific Prometheus metric patterns but the expected patterns did not include labels that were recently added:

• client_ip (added in feat: add clientip and user agent in metrics #19717)
• user_agent (added in feat: add clientip and user agent in metrics #19717)
• model_id (added in feat: Add model_id label to Prometheus metrics (#18048) #19678)

Root Cause

When these PRs added new labels to litellm_proxy_failed_requests_metric and litellm_proxy_total_requests_metric, the test was not updated to match. The test does an exact substring match on the Prometheus metric output, so when the new labels appeared (in alphabetical order between existing labels), the expected pattern no longer matched.

Solution

Update the expected metric patterns in test_proxy_failure_metrics to include the new labels in the correct alphabetical order (as Prometheus outputs them).

Labels updated for litellm_proxy_failed_requests_metric:

api_key_alias, client_ip, end_user, exception_class, exception_status, hashed_api_key, model_id, requested_model, route, team, team_alias, user, user_agent, user_email

Labels updated for litellm_proxy_total_requests_metric:

api_key_alias, client_ip, end_user, hashed_api_key, model_id, requested_model, route, status_code, team, team_alias, user, user_agent, user_email

Testing

This is a test fix - the CI test suite will validate the fix passes.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Error		Feb 1, 2026 0:58am

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[shin-bot-litellm]

Analysis

Failing Test

• Test: test_proxy_failure_metrics in tests/otel_tests/test_prometheus.py
• CI Link: https://app.circleci.com/pipelines/github/BerriAI/litellm/56743/workflows/9f492c05-c440-4c80-84d4-34c6cd566930/jobs/1140648/tests

Root Cause

The test checks for exact Prometheus metric patterns using substring matching. When PRs #19717 and #19678 added new labels (client_ip, user_agent, model_id) to the metrics, the test patterns were not updated.

Prometheus outputs labels in alphabetical order, so the new labels appear in the middle of the existing pattern, breaking the substring match:

Before (expected by test):

...api_key_alias="None",end_user="None",exception_class=...

After (actual output):

...api_key_alias="None",client_ip="None",end_user="None",exception_class=...

Fix

Updated the expected patterns to include all current labels in the correct alphabetical order.

CI Validation

Waiting for CI to run to validate the fix works.

[greptile-apps]

Greptile Overview

Greptile Summary

Updated test_proxy_failure_metrics to include three recently added Prometheus metric labels (client_ip, user_agent, and model_id) in the expected metric patterns. The test was failing because it expected an exact substring match but the labels were added in alphabetical order by Prometheus.

Key changes:

• Updated litellm_proxy_failed_requests_metric_total pattern to include client_ip="None", user_agent="None", and model_id="None" in alphabetical order
• Updated litellm_proxy_total_requests_metric_total pattern to include the same three labels in alphabetical order
• Also includes unrelated test fixes in other files for mock injection and test isolation issues

Confidence Score: 5/5

• Safe to merge - test-only changes that fix failing CI tests
• All changes are test fixes with no impact on production code. The Prometheus metric label updates match the actual metric definitions in the codebase, and other test improvements enhance test isolation and reliability.
• No files require special attention

Important Files Changed

Filename	Overview
tests/otel_tests/test_prometheus.py	Updates expected Prometheus metric label patterns to include client_ip, user_agent, and model_id labels in alphabetical order
tests/test_litellm/enterprise/enterprise_callbacks/send_emails/test_resend_email.py	Improved mock injection by directly setting logger.async_httpx_client to bypass caching issues
tests/test_litellm/test_utils.py	Added test isolation with unique model name, cache clearing, and proper cleanup to prevent test interference

Sequence Diagram

sequenceDiagram
    participant Test as test_proxy_failure_metrics
    participant Proxy as LiteLLM Proxy
    participant Prom as Prometheus Metrics
    
    Test->>Proxy: POST /chat/completions (bad request)
    Proxy->>Prom: Increment litellm_proxy_failed_requests_metric
    Note over Prom: Labels: api_key_alias, client_ip, end_user,<br/>exception_class, exception_status,<br/>hashed_api_key, model_id, requested_model,<br/>route, team, team_alias, user,<br/>user_agent, user_email
    Proxy->>Prom: Increment litellm_proxy_total_requests_metric
    Note over Prom: Labels: api_key_alias, client_ip, end_user,<br/>hashed_api_key, model_id, requested_model,<br/>route, status_code, team, team_alias,<br/>user, user_agent, user_email
    Proxy-->>Test: 429 Rate Limit Error
    Test->>Proxy: GET /metrics
    Proxy->>Prom: Export metrics
    Prom-->>Proxy: Formatted metrics (alphabetically sorted labels)
    Proxy-->>Test: Metrics response
    Test->>Test: Assert metric patterns match (including new labels)

Loading

[greptile-apps]

_{3 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}