feat(prometheus): add api_provider label to token, latency, request and cache metrics

[shivijain2323]

Relevant issues

LIT-4178

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Before

After

Type

🐛 Bug Fix

Changes

A subset of Prometheus metrics carried no api_provider label even though sibling metrics emitted from the same call sites (litellm_spend_metric, litellm_requests_metric) already did, so there was no way to break token usage, latency, request counts or cache hits down by upstream provider. This adds api_provider to the token metrics (litellm_input_tokens_metric, litellm_output_tokens_metric, litellm_total_tokens_metric), the latency metrics (litellm_llm_api_latency_metric, litellm_llm_api_time_to_first_token_metric, litellm_request_total_latency_metric, litellm_request_queue_time_seconds), the proxy request counters (litellm_proxy_total_requests_metric, litellm_proxy_failed_requests_metric) and the cache metrics.

The cache label is added to the shared _cache_metric_labels list, so the change covers all five metrics that use it: litellm_cache_hits_metric, litellm_cache_misses_metric, litellm_cached_tokens_metric, litellm_provider_cache_read_input_tokens_metric and litellm_provider_cache_creation_input_tokens_metric. The label-presence test asserts every one of them.

On the success path no new plumbing is needed; enum_values.api_provider is already populated from standard_logging_payload["custom_llm_provider"], so the value flows through as soon as the label is in the metric's list. For the client-side failure path, where a request can fail before a deployment resolves, the provider is derived best-effort from litellm_params.custom_llm_provider, then a partial standard_logging_object, then inference from the requested model name via litellm.get_llm_provider, falling back to empty rather than guessing.

[shivijain2323]

@greptileai

[greptile-apps]

Greptile Summary

This PR adds the api_provider label to nine metric families in the Prometheus integration — token, latency, proxy request-count, and cache metrics — which previously could not be broken down by upstream provider even though the value was available on every payload. The failure path gains a new _extract_api_provider_from_request_data helper with a priority-ordered fallback chain; the success path required no new plumbing because enum_values.api_provider was already populated there.

• Label additions (litellm/types/integrations/prometheus.py): API_PROVIDER appended to ten metric label lists, including the shared _cache_metric_labels list, which fans out to all five cache metric families.
• Failure-path plumbing (litellm/integrations/prometheus.py): New static helper _extract_api_provider_from_request_data resolves the provider from litellm_params, a partial standard_logging_object, or model-name inference via litellm.get_llm_provider, with narrowly scoped exception handling and debug-level logging for unexpected errors.
• Tests (tests/test_litellm/integrations/test_prometheus_labels.py): Four new unit tests covering label-list assertions for all 14 affected metrics, factory value-flow checks, and exception-handling behavior of the helper — all mock-only, no network calls.

Confidence Score: 5/5

Safe to merge — the change is additive (new label on existing metrics), the success path was already wired correctly, the failure path's new helper is narrowly scoped and never raises, and every affected metric family is covered by the new tests.

All three changed files are coherent: the label lists, the runtime plumbing, and the tests agree on the same set of 14 metric families. The success path already carried api_provider in enum_values (line 1206 of prometheus.py), so no regression risk there. The new _extract_api_provider_from_request_data helper handles the failure path safely with priority-ordered fallbacks and well-scoped exception catching. The four new tests are all mock-only and cover label presence, factory value flow, and exception behavior. No pre-existing logic was altered; previously discussed concerns (time-series discontinuity, broad exception catch) have been addressed by the author.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/types/integrations/prometheus.py	Adds API_PROVIDER to ten metric label lists (nine discrete lists plus the shared _cache_metric_labels); all five cache metrics that alias that shared list now carry the label, which is explicitly asserted in the new tests.
litellm/integrations/prometheus.py	Adds _extract_api_provider_from_request_data static helper and plumbs its result into UserAPIKeyLabelValues in async_post_call_failure_hook; success path already carried api_provider in enum_values from standard_logging_payload["custom_llm_provider"].
tests/test_litellm/integrations/test_prometheus_labels.py	Four new unit tests added: label-list presence for all 14 affected metrics, label factory flow, helper function priority-chain logic, and exception-handling (BadRequestError vs unexpected errors). All mock-only, no network calls.

_{Reviews (4): Last reviewed commit: "feat(prometheus): add api_provider label..." | Re-trigger Greptile}

[greptile-apps]

Greptile Summary

This PR adds the api_provider label to eleven Prometheus metrics that were previously missing it — token counters (input/output/total), latency histograms (API latency, TTFT, total request latency, queue time), request counters (total and failed), and cache hit/miss metrics. On the failure path where the provider is not yet resolved, a new _extract_api_provider_from_request_data helper is introduced to perform best-effort inference from litellm_params, standard_logging_object, and finally from the model name via litellm.get_llm_provider.

• Label plumbing: The success path correctly reads api_provider from standard_logging_payload["custom_llm_provider"] (already available at call time), and the new helper covers the failure path with a safe fallback to None when the provider cannot be determined.
• Shared list side-effect: Adding API_PROVIDER to _cache_metric_labels also propagates the label to litellm_cached_tokens_metric, litellm_provider_cache_read_input_tokens_metric, and litellm_provider_cache_creation_input_tokens_metric, which share the same list — these three are not mentioned in the PR description or tested.
• Tests: Three new unit tests check label presence, end-to-end value flow through the label factory, and all branches of the failure-path inference logic without making network calls.

Confidence Score: 4/5

Safe to merge; the label additions are additive and the new helper degrades gracefully to None on any error.

The core logic is correct and well-tested. The two items that warrant a second look are: the bare except Exception in _extract_api_provider_from_request_data that silently discards unexpected errors from get_llm_provider, and the undocumented propagation of api_provider to three additional cache metrics through the shared _cache_metric_labels list — users who have already scraped those series will see a label-set change on upgrade with no warning.

litellm/types/integrations/prometheus.py — the _cache_metric_labels shared list change; litellm/integrations/prometheus.py — the exception handling in _extract_api_provider_from_request_data.

Important Files Changed

Filename	Overview
litellm/integrations/prometheus.py	Adds _extract_api_provider_from_request_data static method and wires api_provider into async_post_call_failure_hook enum_values; success path already had api_provider from standard_logging_payload.
litellm/types/integrations/prometheus.py	Adds API_PROVIDER label to 9 metric label lists; the _cache_metric_labels shared list also propagates api_provider to litellm_cached_tokens_metric, litellm_provider_cache_read_input_tokens_metric, and litellm_provider_cache_creation_input_tokens_metric beyond what the PR description and tests enumerate.
tests/test_litellm/integrations/test_prometheus_labels.py	Adds three new unit tests covering label presence, value plumbing through the label factory, and failure-path provider inference; all use mocks or local lookups with no real network calls.

Comments Outside Diff (1)

1. litellm/types/integrations/prometheus.py, line 685-703 (link)

   api_provider implicitly added to three extra cache metrics via shared list

   Adding API_PROVIDER to _cache_metric_labels propagates to all five metrics that alias this list: litellm_cache_hits_metric, litellm_cache_misses_metric, litellm_cached_tokens_metric, litellm_provider_cache_read_input_tokens_metric, and litellm_provider_cache_creation_input_tokens_metric. The PR description and the new tests only mention the first two. The three additional metrics silently gain the label without documentation or test coverage, which may surprise existing users whose TSDB already has those series recorded without the label dimension. If the intent is to add api_provider to all five, the tests should assert it for all five and the PR description should note the broader scope.

   Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

_{Reviews (2): Last reviewed commit: "feat(prometheus): add api_provider label..." | Re-trigger Greptile}

[shivijain2323]

@greptileai

[shivijain2323]

@greptileai

[yucheng-berri]

Thanks for the PR. One thing I would like to address before merge:

The tests cover labels/helpers, but not the real emit wiring; deleting the production api_provider assignments still passes, could you please add a hook-level test that inspects the collected metric sample.

[shivijain2323]

Thanks for the PR. One thing I would like to address before merge:

The tests cover labels/helpers, but not the real emit wiring; deleting the production api_provider assignments still passes, could you please add a hook-level test that inspects the collected metric sample.

Good catch, thanks. You're right that the earlier tests only covered the label config and the helper, so deleting the real api_provider assignment would have slipped through.

I've added two hook-level tests that run the actual loggers and inspect the collected metric samples:

one drives async_log_success_event and asserts the emitted litellm_total_tokens_metric sample carries api_provider="openai"
one drives async_post_call_failure_hook and asserts the emitted litellm_proxy_failed_requests_metric sample carries api_provider="openai"
To make sure they actually guard the wiring, I ran a quick mutation check: I set both production api_provider assignments to None, and both new tests failed; restoring the real code makes them pass again. So the emit path is now covered, not just the label lists and the helper.