feat(prometheus): emit per-token-type detail metrics (LIT-3220)

[oss-agent-shin]

Summary

Adds five sparse Prometheus counter metrics that break out the token detail fields providers already report in usage.prompt_tokens_details and usage.completion_tokens_details:

Metric	Source
litellm_input_cached_tokens_metric	prompt_tokens_details.cached_tokens (OpenAI prompt cache, Anthropic cache_read_input_tokens)
litellm_input_cache_creation_tokens_metric	prompt_tokens_details.cache_creation_tokens (Anthropic cache_creation_input_tokens)
litellm_input_audio_tokens_metric	prompt_tokens_details.audio_tokens
litellm_output_reasoning_tokens_metric	completion_tokens_details.reasoning_tokens
litellm_output_audio_tokens_metric	completion_tokens_details.audio_tokens

No breaking changes — existing litellm_input_tokens_metric / litellm_output_tokens_metric / litellm_total_tokens_metric are untouched. The new counters are additive and only emit when the underlying value is > 0, keeping scrape output sparse for providers that don't report a given field. Each new counter reuses the same label set as its parent input/output token counter, so dashboards can join cleanly.

Data is read from the canonical Usage dict that get_standard_logging_object_payload already attaches at standard_logging_payload["metadata"]["usage_object"] — no new plumbing through the logging pipeline.

Closes LIT-3220.

Visual proof — /metrics output after driving both an Anthropic-shaped and an OpenAI-shaped Usage payload

For the Anthropic call (cached_tokens=800, cache_creation_tokens=200, no audio/reasoning) and the OpenAI audio call (cached_tokens=100, audio_in=50, reasoning=220, audio_out=40), the new counters appear with the expected values and stay sparse for fields the provider didn't report (no cache_creation line for the OpenAI call, no audio/reasoning lines for the Anthropic call):

The same artifact is committed at proof/lit-3220-metrics.png on this branch.

Test plan

• 10 new unit tests in tests/test_litellm/integrations/test_prometheus_token_detail_metrics.py covering registration, label-set parity, all-types increment, and zero/None/negative/missing-metadata skip paths
• Existing test_prometheus_cache_metrics.py / test_prometheus_labels.py / test_prometheus_metric_name_consistency.py continue to pass (33/33 green)
• uv run black . clean
• End-to-end smoke: drove PrometheusLogger._increment_token_metrics with Anthropic- and OpenAI-shaped payloads, scraped via prometheus_client.generate_latest(), confirmed correct values + sparse behavior (see screenshot)

🤖 Generated with Claude Code

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 3 committers have signed the CLA.

✅ yuneng-berri
❌ shin-berri
❌ krrish-berri
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codspeed-hq]

Merging this PR will not alter performance

✅ 16 untouched benchmarks

---

_{Comparing oss-agent-shin:feat/lit-3220-token-detail-metrics (4167491) with main (79b4578)}

[codecov]

Codecov Report

❌ Patch coverage is 95.65217% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/integrations/prometheus.py	94.44%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

Adds five sparse Prometheus counters (litellm_input_cached_tokens_metric, litellm_input_cache_creation_tokens_metric, litellm_input_audio_tokens_metric, litellm_output_reasoning_tokens_metric, litellm_output_audio_tokens_metric) that break out per-token-type detail fields already present in usage.prompt_tokens_details / usage.completion_tokens_details. The change is fully additive — existing total-token counters are untouched and the new counters are only emitted when the underlying value is > 0.

• Five new counters registered in PrometheusLogger.__init__ and incremented via the new _increment_token_detail_metrics helper, which reads from standard_logging_payload[\"metadata\"][\"usage_object\"] — no new plumbing in the hot path.
• Label sets for the new metrics alias the existing litellm_input_tokens_metric / litellm_output_tokens_metric label lists so dashboards can join across them.
• Ten new unit tests cover registration, label-set parity, all-types increment, and zero/None/negative/missing-metadata skip paths; all tests use mocks and make no network calls.

Confidence Score: 4/5

This is a purely additive change with no modifications to existing counters or the request hot path; it is safe to merge.

The new helper reads from a pre-existing metadata field, all counters are sparse (only emitted when the value is > 0), and existing token metrics are unchanged. The isinstance(…, dict) guards inside the detail-metrics list are technically reachable only when a Pydantic wrapper object bypasses the or {} fallback, which silently drops the metric rather than causing an error. The binary PNG committed to the source tree is a process concern with no runtime impact.

The proof/lit-3220-metrics.png binary should not live in the main repo; the isinstance(prompt_details, dict) guards in _increment_token_detail_metrics warrant a second look for the Pydantic-object edge case.

Important Files Changed

Filename	Overview
litellm/integrations/prometheus.py	Registers five new sparse Prometheus counters for per-token-type detail metrics and adds _increment_token_detail_metrics; the implementation is clean and follows the existing _inc_labeled_counter pattern correctly.
litellm/types/integrations/prometheus.py	Adds five new metric names to the DEFINED_PROMETHEUS_METRICS Literal and assigns label sets in PrometheusMetricLabels by aliasing the existing input/output label lists; straightforward and consistent with the existing pattern.
tests/test_litellm/integrations/test_prometheus_token_detail_metrics.py	New unit test file with 10 mock-only tests covering registration, label parity, zero/None/negative skip paths, and full increment; tests correctly bind to the real _increment_token_detail_metrics implementation via unbound method call.
proof/lit-3220-metrics.png	Binary screenshot committed into the main repo; documentation artifacts like this belong in the litellm-docs repo or hosted as PR attachments, not in the source tree.

Comments Outside Diff (1)

1. proof/lit-3220-metrics.png, line 1 (link)

   Binary proof artifact committed to main repo

   Per the team's documentation guidelines, artifacts like screenshots and visual proof should live in the litellm-docs repo or be attached directly to the PR (GitHub renders images pasted or dragged into PR descriptions without requiring a committed file). Committing a binary PNG to the source tree inflates repo clone size for every developer and CI runner going forward, with no runtime benefit.

   Rule Used: Prevent documentation from being added - needs to ... (source)

_{Reviews (1): Last reviewed commit: "feat(prometheus): emit per-token-type de..." | Re-trigger Greptile}

[greptile-apps]

isinstance(…, dict) guards are unreachable after the or {} fallback

prompt_details and completion_details are assigned via … or {}, which means they are either a plain dict (from model_dump() or JSON parsing) or a truthy non-dict value such as a PromptTokensDetailsWrapper / CompletionTokensDetailsWrapper Pydantic object. In the Pydantic-object case isinstance(…, dict) returns False and every field silently resolves to None, so no detail metric fires — even though both wrapper classes expose a compatible .get() method. Using .get() directly (which works for both dicts and the wrapper models) after the or {} guard would be both simpler and more defensive.