fix(otel): export v2 gen_ai client metrics to the configured meter provider

[yassin-berriai]

Relevant issues

Resolves LIT-3789

Linear ticket

Resolves LIT-3789

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Root cause

With LITELLM_OTEL_V2=true and LITELLM_OTEL_INTEGRATION_ENABLE_METRICS=true, the V2 OpenTelemetry integration records the six gen_ai.client.* histograms but they never reach an operator's metric exporter. An earlier read of this suspected plumbing/metrics.py was dead code and the meter provider was never referenced; that is inaccurate. logger.py imports create_genai_metrics/GenAIMetricRecorder/build_meter_provider, calls _init_metrics in __init__, and records via _record_metrics in the success path, so the recording code does run.

The real defect is that _init_metrics built a brand-new MeterProvider locally via build_meter_provider and recorded the histograms into it without ever publishing it. That provider is disconnected from the global meter pipeline the operator configures, so the GenAI histograms land somewhere the operator cannot see while the FastAPI server metrics, which bind to the global meter provider, show up fine. This contrasts with the trace side, where the proxy publishes the V2 TracerProvider as the OTel global so server spans and gen-ai spans share one provider.

This builds directly on #30326 (merged), which introduced the six gen_ai.client.* histograms and the private MeterProvider. That PR made the metrics record; this one makes them reach the operator's exporter

Fix

Resolve the meter provider the OTel-idiomatic way in providers.resolve_meter_provider. An injected meter_provider still wins for DI and tests. Otherwise the operator's globally configured MeterProvider is reused when one is set, so its readers and exporters receive the GenAI histograms alongside the server metrics; when none is set, V2 builds one from the config and registers it as the global so it owns metrics export, mirroring how V2 owns trace export today. The logger now obtains its meter from that resolved provider with the LiteLLM scope version stamped, consistent with get_tracer.

Changes

• plumbing/providers.py: add resolve_meter_provider (reuse global, else build and register) and a get_meter helper that stamps the LiteLLM scope version
• logger.py: _init_metrics resolves the provider through resolve_meter_provider instead of always building a private one
• tests/test_litellm/integrations/otel/test_otel_v2_metrics.py: regression test proving the histograms reach an operator-configured global MeterProvider when no provider is injected
• integrations/otel/README.md: document the metric provider resolution

Screenshots / Proof of Fix

Live proxy on port 4001 with V2 OTel and GenAI metrics on. An operator-owned global MeterProvider with a ConsoleMetricExporter is installed at interpreter startup via a sitecustomize.py (the way an application owns metrics), and a real OpenAI gpt-4o-mini chat completion is driven through /v1/chat/completions.

Launch (identical before and after):

export PYTHONPATH=/tmp/otel_bootstrap:$PWD
export OPERATOR_CONSOLE_METRICS=1          # sitecustomize sets a global console MeterProvider
export LITELLM_OTEL_V2=true
export LITELLM_OTEL_INTEGRATION_ENABLE_METRICS=true
export OTEL_EXPORTER=otlp_http
export OTEL_ENDPOINT=http://127.0.0.1:9999  # keep the pre-fix isolated provider off the operator console
python litellm/proxy/proxy_cli.py --config proof.yaml --port 4001 --num_workers 1

curl -s http://localhost:4001/v1/chat/completions \
  -H "Content-Type: application/json" -H "Authorization: Bearer sk-1234" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Say hello in 3 words"}]}'
# -> {"id":"chatcmpl-...","model":"gpt-4o-mini","choices":[{"message":{"content":"Hello, how are you?"...

Metric names seen in the operator's console export before the fix; the FastAPI server metrics arrive but no gen_ai.client.*:

   3 "name": "http.server.active_requests"
   3 "name": "http.server.duration"
   3 "name": "http.server.request.size"
   3 "name": "http.server.response.size"
   3 "name": "opentelemetry.instrumentation.fastapi"
gen_ai.client count: 0

Same operator console after the fix; the gen_ai.client.* histograms now arrive under the litellm scope alongside the server metrics:

   1 "name": "gen_ai.client.operation.duration"
   1 "name": "gen_ai.client.response.duration"
   1 "name": "gen_ai.client.token.cost"
   1 "name": "gen_ai.client.token.usage"
   2 "name": "http.server.active_requests"
   1 "name": "http.server.duration"
   1 "name": "http.server.request.size"
   1 "name": "http.server.response.size"
   1 "name": "litellm"
   2 "name": "opentelemetry.instrumentation.fastapi"
gen_ai.client count: 10

One exported gen_ai.client.operation.duration data point from the operator console after the fix:

{
  "scope": "litellm",
  "metric": {
    "name": "gen_ai.client.operation.duration",
    "description": "GenAI operation duration",
    "unit": "s",
    "data": {
      "data_points": [
        {
          "attributes": {
            "gen_ai.operation.name": "chat",
            "gen_ai.system": "openai",
            "gen_ai.request.model": "gpt-4o-mini",
            "gen_ai.framework": "litellm"
          }
        }
      ]
    }
  }
}

Type

🐛 Bug Fix

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

This PR fixes a bug where GenAI gen_ai.client.* histograms were silently dropped by recording into a private, disconnected MeterProvider instead of the operator's globally configured one, so the histograms never reached the operator's readers or exporters.

• providers.resolve_meter_provider: new helper that returns an injected provider (DI/tests), or reuses the operator's existing global SDKMeterProvider/NoOpMeterProvider, or builds one from config and registers it as the global — mirroring how V2 already owns trace export.
• logger._init_metrics: one-line change to call resolve_meter_provider + get_meter instead of always calling build_meter_provider, routing histograms into the operator's pipeline.
• Three targeted regression tests cover the operator-global, injected, and NoOp paths with no real network calls.

Confidence Score: 5/5

Safe to merge — the change is narrowly scoped to meter provider resolution, is well-tested, and does not touch the request critical path or authentication layer.

The fix is a straightforward one-level indirection: instead of always constructing an isolated MeterProvider, the code now checks the OTel global first. The logic is simple, the three new tests cover the injected, global-reuse, and NoOp paths, and the PR includes live proxy proof that histograms reach the operator's exporter. No auth, DB, or routing code is touched.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/integrations/otel/plumbing/providers.py	Adds resolve_meter_provider (reuse global SDK/NoOp provider or build+register a new one) and a get_meter helper that stamps the LiteLLM scope version; build_meter_provider return type tightened to SDKMeterProvider.
litellm/integrations/otel/logger.py	_init_metrics now calls resolve_meter_provider + get_meter instead of build_meter_provider + provider.get_meter; imports updated accordingly.
tests/test_litellm/integrations/otel/test_otel_v2_metrics.py	Three new regression tests added: operator-configured global provider receives histograms, injected provider is preferred verbatim, and an explicit NoOpMeterProvider is not overridden.
litellm/integrations/otel/README.md	Minor update to the metrics.py bullet in the plumbing overview to describe the new provider resolution logic.

_{Reviews (4): Last reviewed commit: "fix(otel): type resolve_meter_provider a..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 76.47059% with 4 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/integrations/otel/plumbing/providers.py	73.33%	4 Missing ⚠️

📢 Thoughts on this report? Let us know!

[yassin-berriai]

@greptileai

[yassin-berriai]

@greptileai

[yassin-berriai]

@greptileai

[mateo-berri]

LGTM; thanks!