feat(proxy): push-based OTLP billable-request metering for enterprise deployments

[yassin-berriai]

Relevant issues

Resolves LIT-4089

Linear ticket

LIT-4089

Design doc: https://app.notion.com/p/38e43b8acdab81feb2f3da4a63840bd5

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Live proxy with a real LITELLM_LICENSE, exporting OTLP/HTTP to a local replica of the global collector (same receiving pipeline as production, which terminates TLS at its load balancer before the collector container).

Traffic matrix: 3x /v1/chat/completions, 1x /v1/messages, 1x /v1/embeddings (all 2xx), plus controls that must not bill: GET /v1/models 200, /health/readiness 200, bad-model 400, GET /v1/videos 400.

Collector received exactly the successful inference calls:

litellm.enterprise.billable_requests
  http.route=/chat/completions  value=3
  http.route=/messages          value=1
  http.route=/embeddings        value=1

UI usage parity (/user/daily/activity, the source of the dashboard usage page): successful_requests: 5, failed_requests: 2. The collector total (5) matches the UI successful requests exactly, and the 2 failures are excluded on both sides.

Metering-bypass fix, before and after on the same live setup (1x chat + 1x /v1/messages, both 200):

before: http.route=/chat/completions value=1          (/v1/messages missing)
after:  http.route=/chat/completions value=1
        http.route=/messages         value=1

Recorder-init fix verified live: with LITELLM_LICENSE and all billing env vars provided only through the YAML config's environment_variables (nothing in the shell), the collector receives both routes. On the previous code this scenario exported nothing because the recorder was built at import time.

Review-round fixes verified against the production collector at telemetry.litellm.ai, using a real mTLS leaf issued from the CA Service pool. Getting there surfaced and fixed an infra bug: certs issued by the onboarding workflow carried no extensions, and the load balancer rejects leaves without the clientAuth EKU (client_cert_chain_invalid_eku in the LB logs, visible only as a post-handshake connection reset to the client); the workflow now requests key_usages digital_signature and extended_key_usages client_auth

Traffic on the live proxy (port 4159, license + billing config via env, gpt-5.4-nano): POST /v1/chat/completions 200 bills /chat/completions; POST /v1/messages 200 bills /v1/messages; POST /v1/threads then POST /v1/threads/{id}/messages both 200 and neither bills nor writes a SpendLogs row (the route the old /messages suffix over-billed); POST /openai/v1/assistants 200 bills /openai and writes a pass_through_endpoint SpendLogs row; DELETE /openai/v1/assistants/{id} 200 does not bill (POST gate)

LiteLLM_SpendLogs finished with 7 success rows; 6 were billable under the classifier, and the single gap is the DELETE passthrough, the POST gate's deliberate undercount. The collector logged the proxy's OTLP pushes at the 5s export interval:

gcloud logging read 'resource.labels.service_name="litellm-tel-collector" AND httpRequest.requestUrl:"/v1/metrics"'
2026-07-09T07:57:49Z  200  https://telemetry.litellm.ai/v1/metrics
2026-07-09T07:57:44Z  200  https://telemetry.litellm.ai/v1/metrics

The mTLS gate itself was exercised negatively too: a leaf without the clientAuth EKU and a connection with no client certificate are both cut off at the edge and never reach the collector

Deployment surface

A successful recorder build now emits one info line naming the collector endpoint and the export interval. Every disable path already logged; without a positive signal an operator could not tell a metering component from one that silently returned None because it never saw the license, and a component can carry the cert mount and the billing env and still meter nothing

The cert env vars (LITELLM_BILLING_METRICS_CLIENT_CERT, _CLIENT_KEY, _CA_CERT) now accept either a filesystem path or the PEM itself. Inline PEM, recognized by the -----BEGIN prefix, is written once at recorder build into a 0700 temp dir as a 0600 file, and the exporter is pointed at that path. This unblocks secret stores that inject values as environment content and cannot mount them as files, such as ECS reading Secrets Manager. Paths keep working untouched, and the mixed case works too, so a deployment can mount the CA while injecting the client credentials. A write failure disables metering through the existing failure-as-None path rather than raising

The componentized chart at helm/litellm grows a top-level billingMetrics block, off by default, consumed by the gateway and the backend. Applying each component's route allowlist to the proxy app splits the billable surface 75 routes to the gateway and one to the backend, /{mcp_server_name}/mcp, the named-server MCP transport, which writes a SpendLogs row; metering only the gateway would silently drop that traffic. The migrations job gets neither the env nor the cert. When enabled it renders the endpoint plus the two cert paths and mounts secretName read-only at /etc/litellm/billing-mtls. secretName defaults to litellm-billing-metrics-mtls, the conventional name, so the common path is to create that Secret with tls.crt and tls.key and set enabled: true; caSecretName is optional and only needed for private collectors whose server certificate is not on the public web PKI. Enabling it without a secretName, or with an empty endpoint, fails the render with a named message instead of producing a gateway that silently never exports. The generic gateway.volumes / gateway.volumeMounts / gateway.extraEnv paths are untouched and still compose with it

The helm/litellm-helm chart and the terraform/litellm/aws and terraform/litellm/gcp reference templates gain the same switch. The terraform templates take the certificate and key as PEM variables, store them in Secrets Manager or Secret Manager, and inject them as environment values gated on billing_metrics_endpoint being non-empty, so the default deployment is unchanged. A lifecycle precondition on the gateway task definition and the gateway Cloud Run service requires the client certificate and its key together whenever the endpoint is set, so a half-configured metering block fails the plan rather than deploying a proxy that logs "missing config" and never exports. billing_metrics_ca_cert_pem stays optional and empty falls back to the system trust store

Performance

Metering costs roughly one percent of throughput at the shipped default, and the cost is in the background export thread rather than the per-request path. That is small enough to ship and large enough not to call it free.

Sequential A/B on a dev machine drifts far more than the effect being measured, so both variants ran simultaneously: two worktrees (the staging merge-base and this branch), a proxy each on its own port with its own Postgres, one shared local mock upstream so neither side touches a real provider, identical closed-loop drivers at concurrency 32 started on a shared wall clock, and 20 seconds of warmup before 60 to 75 seconds of steady state. The candidate ran with a real license and a real client certificate, exporting to the production collector, verified two ways: a probe confirmed the recorder was constructed and a real mutual-TLS flush succeeded, and a negative control with a bad certificate path produced the expected disabled warning. That check matters because the recorder returns silently when a deployment is not licensed, so quiet logs alone prove nothing.

Two harness effects had to be removed before the numbers meant anything. Swapping only ports and databases left process start order fixed, and an A/A control running identical code on both sides showed the second-launched slot was about 1.2 percent faster on that alone. The final design is a balanced two by two over start order and port assignment, with A/A null rounds interleaved between metering rounds so machine drift hits both equally; it drifted a lot, with baseline throughput ranging from 456 to 320 RPS as other work loaded the machine. Reading a single round is therefore meaningless: the A/A null, identical code on both sides, spans -2.02 to +2.14 percent.

At the default 60000 ms export interval, eight metering rounds against eight interleaved A/A null rounds:

arm	n	mean delta	sd
metering vs baseline	8	-0.71 percent	0.76
A/A null, identical code	8	+0.39 percent	1.32

Subtracting the null gives a -1.10 percent effect, 95 percent confidence interval -2.15 to -0.04 percent. Zero errors in 866618 requests. Latency moved by +0.54 ms at p50 and +1.9 ms at p99, against a null of -0.07 ms and -0.9 ms.

The per-request work is not where the cost lives. Microbenchmarked in process, classify_billable_request takes 0.56 us per call and recorder.record takes 1.03 us, together 1.59 us against a 2273 us per-request budget at 440 RPS, which is 0.07 percent and roughly fifteen times too small to explain the measured effect. What does track the effect is export frequency, which only the background PeriodicExportingMetricReader thread drives: at a 5000 ms interval, twelve times more often than any real deployment, the cost roughly doubles. The overhead is GIL and CPU contention from the periodic mutual-TLS OTLP export, it shrinks as the interval lengthens, and the shipped default sits at the lenient end. Anyone lowering LITELLM_BILLING_METRICS_EXPORT_INTERVAL_MS should expect to pay for it.

One control was intended and not obtained: metering active with the export interval set high enough that the exporter never fires inside the measurement window, which would separate per-request cost from exporter-thread cost directly. The rig was torn down before that round produced data, so the attribution rests on two converging measurements, the microbenchmark and the export-frequency scaling, rather than on that direct control

These numbers were captured with the candidate at 91f5e26 and the baseline at merge-base e84a19a. The branch has advanced since, but nothing in billable_request_metrics_middleware.py or the proxy's middleware stack changed after that commit; the later work is helm, terraform, docs, and one info log emitted once when the recorder is built, so the per-request path the numbers describe is the one that ships

To reproduce: stand up the two worktrees against separate databases and a shared mock upstream, drive both at once with identical generators, balance start order as well as port assignment, interleave A/A null rounds, and compare arm means rather than single rounds

Type

🆕 New Feature

Changes

Push-based, license-gated request metering for enterprise deployments, using OpenTelemetry as the open standard.

BillableRequestMetricsMiddleware (pure ASGI) reads the final status and the x-litellm-model-id response header off http.response.start, classifies the path into llm, mcp, or a2a, and records one count per 2xx via an injected recorder. LLM classification covers the full inference surface that writes a SpendLogs row on success (chat/completions, completions, embeddings, responses, rerank, moderations, images generations/edits/variations, audio, /v1/messages, videos and remix, /v1/ocr, Gemini generateContent/streamGenerateContent, /v1/search and vector-store search, rag ingest/query, and provider passthrough prefixes read from LiteLLMRoutes.mapped_pass_through_routes with /langfuse excluded as observability-only) and is gated to POST so GET reads on the same resources do not bill. MCP and A2A bill only their protocol transport surface: MCP counts the /mcp streamable-HTTP and SSE sub-app plus the /{server}/mcp and /toolset/{name}/mcp aliases (whose handlers rewrite the ASGI scope after the middleware has classified the original path), and A2A counts only the /message/send invoke route, never the bare /a2a/{agent_id} JSON-RPC endpoint whose method is only visible in the request body; the /v1/mcp management API, /v1/a2a/discover, and the .well-known agent-card reads are discovery and administration, not billable traffic. Known limitations, all in the undercount direction: /v1/realtime is a websocket flow the HTTP middleware does not observe; a message/send sent to the bare /a2a/{agent_id} endpoint does not bill, because that path also serves the non-logging task RPCs; management writes that also log spend (batch, file, and fine-tuning job creation, interaction cancel) and non-POST calls that log (passthrough reads and resource GETs) do not bill under the POST inference gate. Whether batch creation should bill is left as a follow-up product decision since batches carry real downstream cost. The exported count is a deliberate lower bound on the dashboard's successful-request count: it matches exactly for the core inference traffic mix, and every known divergence undercounts rather than overcounts. With no recorder the middleware is a transparent pass-through.

BillingMetricsRecorder owns a dedicated OTEL MeterProvider and an OTLP/HTTP exporter that presents a TLS client certificate. The global collector runs behind a load balancer that terminates mutual TLS and validates the client certificate against our CA, then forwards the verified subject with the request; the collector stamps that as the deployment identity. Server verification uses the system trust store since the collector presents a public web-PKI certificate. The recorder is deliberately separate from the global meter provider so a customer's own OTEL metrics are never touched and ours never leak into their backend.

The recorder is resolved on the first request through a factory, after proxy_startup_event has loaded the YAML config's environment_variables, so licenses and certificates supplied that way are honored.

Identity is carried by the TLS client certificate, never the payload; the secret license key is not exported. Only the license org id travels as a resource attribute for a tamper-evidence cross-check at the collector. Certificates are issued per deployment from our CA.

Configuration via env: LITELLM_BILLING_METRICS_ENDPOINT (https URL of the collector), LITELLM_BILLING_METRICS_CLIENT_CERT, LITELLM_BILLING_METRICS_CLIENT_KEY, optional LITELLM_BILLING_METRICS_CA_CERT (only for private or test collectors), optional LITELLM_BILLING_METRICS_EXPORT_INTERVAL_MS.

The collector gateway, CA, and backend wiring are infrastructure tracked separately; this PR is only the litellm-side instrumentation

Review-round hardening (6f12dda): /v1/messages is matched exactly instead of by suffix, so the OpenAI Assistants /v1/threads/{id}/messages route (2xx but no SpendLogs row; a_add_message never dispatches success handlers) no longer bills; Google Interactions creates (/interactions, /v1beta/interactions) bill via a new exact-match list; recorder.record() failures are logged and never propagate to the already-served response; lazy recorder resolution is double-checked under a lock so concurrent first requests build at most one MeterProvider; and empty-string billing env vars disable metering the same as unset ones. Two later review rounds added: POST /mcp-rest/tools/call bills as MCP (it fires the same tool-call spend logging as the /mcp transport) while /mcp-rest list and test endpoints stay excluded; and proxy_shutdown_event now flushes the recorder through a one-slot registry so restarts no longer drop up to one export interval of buffered counts. A final round (382d11b) reverted a short-lived attempt to bill bare POST /a2a/{agent_id}: that route multiplexes JSON-RPC methods off the body, and only message/send and message/stream write a SpendLogs row, while tasks/get, tasks/cancel and the pushNotificationConfig RPCs are forwarded upstream and write none, so billing by path would have overcounted task polling. The same round moved the middleware inside InFlightRequestsMiddleware; it records after the inner app returns, so registered outside it a request could be counted as drained while its record() was still pending and proxy_shutdown_event could stop the exporter underneath it. Passthrough billing deliberately stays prefix-based: successful passthrough POSTs write SpendLogs rows through the StandardPassThroughResponseObject fallback in success_handler.py regardless of subpath, so narrowing to known inference subpaths would undercount against the dashboard

Docs: user-facing documentation for this feature (the metering page, the five env vars in the config settings reference, and the Helm/Terraform deploy recipes) is in BerriAI/litellm-docs#571. A follow-up commit here (c6bb384) also documents mcp_rpm_limit in the update_team docstring, which fixes the documentation CI failure on this PR. That failure is order-dependent flake, not something this branch introduced: test_api_docs.py walks management_endpoints and lets the last file win, and model_management_endpoints.py defines an update_team shim with no annotated params that trivially passes when it shadows the real endpoint. When the filesystem walk order puts team_endpoints.py last, the real docstring is checked and has been missing mcp_rpm_limit since the field was added to UpdateTeamRequest in June; the same failure shows up flakily on other open PRs. The docstring fix makes the check pass in either order

---

Note

Medium Risk
Touches the global proxy middleware stack and enterprise billing export with mTLS credentials; misconfiguration is mostly fail-safe, but metering accuracy and ~1% throughput overhead depend on route classification and export interval.

Overview
Adds enterprise billable-request metering: licensed proxies count successful 2xx traffic on LLM, MCP, and A2A routes and push an OTLP/HTTP counter to LiteLLM’s collector using mTLS (deployment identity from the client cert, not the license key in the payload).

Proxy runtime: New BillableRequestMetricsMiddleware classifies paths conservatively (POST-gated LLM inference aligned with SpendLogs, MCP/A2A transport only) and records via an injected recorder. billing_metrics.py builds a dedicated OTEL MeterProvider, supports cert env vars as file paths or inline PEM, fails open on misconfig, and flushes on shutdown. The recorder is built on the first request via a factory (after YAML environment_variables load) and sits inside InFlightRequestsMiddleware so shutdown does not drop in-flight counts.

Deploy surfaces: Opt-in billingMetrics in both Helm charts (litellm-helm proxy + componentized litellm gateway and backend for named-server MCP); AWS/GCP Terraform inject PEMs from Secrets Manager / Secret Manager with plan-time preconditions when the endpoint is set without cert+key. Docs and helm unit tests cover defaults, CA override, env ordering, and migrations job exclusion.

Minor: mcp_rpm_limit documented on team update API / OpenAPI schema.

^{Reviewed by Cursor Bugbot for commit 6d0a044. Bugbot is set up for automated code reviews on this repo. Configure here.}

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[yassin-berriai]

@greptileai

[greptile-apps]

Greptile Summary

This PR adds enterprise push-based OTLP request metering: a new pure ASGI middleware (BillableRequestMetricsMiddleware) classifies inbound requests into LLM/MCP/A2A categories, counts 2xx responses, and a dedicated recorder pushes them to LiteLLM's collector over mutual TLS using its own isolated MeterProvider. The recorder is resolved lazily on the first request (after proxy_startup_event has loaded YAML env-vars) and shut down via a one-slot registry in proxy_shutdown_event. Helm, Terraform/AWS, and Terraform/GCP modules grow opt-in billingMetrics blocks with validation guards.

• Middleware (billable_request_metrics_middleware.py): pure ASGI, POST-gated for LLM, method-agnostic for MCP/A2A, transparent when no recorder is injected, with a thread-safe double-checked locking factory.
• Recorder (billing_metrics.py): handles inline PEM (writes temp files once at build time), lazy resolution, and a bounded-timeout flush on shutdown via shutdown_billing_metrics_recorder.
• Deploy: Helm and Terraform modules gate metering on the endpoint variable; a Terraform lifecycle precondition rejects half-configured setups at plan time.

Confidence Score: 5/5

Safe to merge. The middleware is a transparent pass-through on unlicensed deployments and its registration order is verified by a test against the live app.

All disable paths return None without touching the request path. The lazy factory, double-checked locking, and shutdown flush are thoroughly tested. Terraform lifecycle preconditions prevent half-configured billing at plan time. The one non-blocking observation is that inline-PEM temp files persist after process exit, which is harmless in containerised deployments.

No files require special attention; billing_metrics.py has the temp-dir cleanup note but poses no correctness risk.

Important Files Changed

Filename	Overview
litellm/proxy/middleware/billable_request_metrics_middleware.py	New pure-ASGI middleware: well-tested classification logic, thread-safe lazy factory, transparent pass-through when recorder is None
litellm/proxy/enterprise_billing/billing_metrics.py	Recorder factory with mTLS exporter, inline-PEM temp-file materialization, and shutdown registry; temp dir is never explicitly cleaned up on shutdown
litellm/proxy/proxy_server.py	Middleware registration inside InFlightRequestsMiddleware and shutdown flush are correct; lazy lambda captures module globals at call time
tests/test_litellm/proxy/enterprise_billing/test_billing_metrics.py	Comprehensive mock-only tests covering all disable paths, inline PEM materialization, shutdown idempotency, and log safety
tests/test_litellm/proxy/middleware/test_billable_request_metrics_middleware.py	Thorough classification parametric tests, concurrency factory test, and middleware ordering assertion against the live app
terraform/litellm/aws/ecs.tf	Billing metrics added to gateway and backend (not migrations), lifecycle precondition correctly rejects endpoint-without-cert at plan time
terraform/litellm/aws/secrets.tf	Secrets Manager resources count-gated on per-PEM flags; recovery_window=0 is intentional for automation
helm/litellm/templates/_helpers.tpl	Billing metrics helpers with required-field validation on secretName and endpoint; CA secret optional
helm/litellm/values.yaml	Opt-in billingMetrics block defaulting to disabled with sensible defaults

_{Reviews (18): Last reviewed commit: "chore(ui): regenerate schema.d.ts for up..." | Re-trigger Greptile}

[greptile-apps]

Greptile Summary

This PR adds enterprise-only OTLP billable-request metering for the LiteLLM proxy. The main changes are:

• New ASGI middleware for classifying billable LLM, MCP, and A2A requests.
• New billing metrics recorder with a dedicated OpenTelemetry meter provider and mTLS OTLP exporter.
• Proxy wiring that injects the recorder into the middleware for licensed deployments.
• Unit tests for route classification, recording behavior, config loading, mTLS credential mapping, and metric attributes.

Confidence Score: 4/5

The metering feature is mostly isolated, but the recorder initialization order can leave supported config-based licensed deployments unmetered.

The changed middleware and recorder are covered by focused tests, and the main remaining risk is concentrated in startup wiring for config-loaded environment variables.

litellm/proxy/proxy_server.py

T-Rex Logs

What T-Rex did

• Ran a focused Python harness to reproduce the import-time absence of a recorder and to verify that billing configuration loaded from environment variables enables a recorder afterwards.
• Validated that there was no billable_request_metrics_middleware.py initially, and after the change the middleware existed and began recording calls only for LLM 200, MCP 201, and A2A 204, with the total remaining 3 after health 200 and LLM 404.
• Compared pre-change and post-change OTLP recorder logs and verified the post-change setup uses BillingMetricsRecorder with TLS credentials, an exporter endpoint, and the expected resource attributes.

_{Ran code and verified through T-Rex}

Comments Outside Diff (1)

1. litellm/proxy/proxy_server.py, line 1761-1764 (link)

   Recorder built too early
   build_billing_metrics_recorder runs when proxy_server.py is imported, before _load_environment_variables() can apply environment_variables from config.yaml and before premium_user is refreshed there. A licensed deployment that sets LITELLM_LICENSE or the new billing cert env vars in config will keep recorder=None, so all billable requests are silently unmetered for that supported configuration path.

   Rule Used: What: avoid backwards-incompatible changes without... (source)

   Artifacts

   
   

   Repro: focused runtime harness exercising proxy import then config environment variable loading

   • Contains supporting evidence from the run (text/x-python; charset=utf-8).

   Repro: execution log showing import-time recorder None and middleware recorder still None after config-only billing env load

   • Keeps the command output available without making the summary code-heavy.

   

   _{Ran code and verified through T-Rex}

_{Reviews (1): Last reviewed commit: "feat(proxy): push-based OTLP billable-re..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 76.44788% with 61 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...itellm/proxy/enterprise_billing/billing_metrics.py	59.28%	57 Missing ⚠️
litellm/proxy/proxy_server.py	75.00%	4 Missing ⚠️

📢 Thoughts on this report? Let us know!

[veria-ai]

PR overview

This PR adds push-based OTLP metering for enterprise proxy deployments, with middleware that classifies successful proxy requests and exports billable-request counters. It also introduces enterprise billing metrics plumbing for emitting those counters from the proxy.

There are still open billing-integrity issues in the request classifier and OTLP export path. Authenticated callers can cause some non-inference passthrough requests to be counted as billable, while A2A streaming calls on the bare agent route can avoid metering; multi-replica deployments may also lose counts because cumulative counters are not uniquely identified per process. Four issues have already been addressed, but the remaining findings can still produce inaccurate enterprise usage accounting.

Open issues (3)

• Medium: Non-inference passthrough requests are billed — litellm/proxy/middleware/billable_request_metrics_middleware.py:82
• Medium: A2A streaming metering bypass — litellm/proxy/middleware/billable_request_metrics_middleware.py:111
• Medium: Replicas export conflicting cumulative counters — litellm/proxy/enterprise_billing/billing_metrics.py:113

Fixed/addressed: 4 · PR risk: 5/10

[yassin-berriai]

@greptileai please review the current head ea2e074

[yassin-berriai]

@greptileai please review the current head dd263fd

[yassin-berriai]

@greptileai please review the current head 6484270

[yassin-berriai]

@greptileai

[yucheng-berri]

bugbot run

[mateo-berri]

Is the veria or bugbot concerns legit?

[veria-ai]

Medium: Non-inference passthrough requests are billed

_PASSTHROUGH_PREFIXES includes broad provider prefixes such as /openai, and this branch records any successful POST below them. An authenticated caller can repeatedly send successful non-inference passthrough requests like POST /openai/v1/assistants or thread/resource creation calls and have them counted as billable LLM requests even though they are not inference calls; classify passthroughs by known inference subpaths instead of the provider prefix alone.

[yassin-berriai]

Checked this against what the proxy actually records, and prefix-based passthrough billing is deliberate; the metering unit here is requests the deployment itself counts as successful on its usage dashboard, not inference calls specifically. pass_through_async_success_handler (litellm/proxy/pass_through_endpoints/success_handler.py) dispatches success handlers for every successful passthrough call: when no provider-specific handler recognizes the subpath it falls back to StandardPassThroughResponseObject and still writes the SpendLogs row; only /langfuse (already excluded from billing) and AssemblyAI non-POST reads opt out. Verified on a live proxy: a 200 POST /openai/v1/assistants wrote a pass_through_endpoint success row in LiteLLM_SpendLogs and counts toward the dashboard successful requests exactly like a chat completion, so excluding it from the exported count would make enterprise metering drift below the number the customer sees on their own usage page, which is the parity invariant this feature is built around. The remaining asymmetry runs the safe direction only: non-POST passthrough calls (GET and DELETE reads) can write SpendLogs rows but never bill because of the POST gate, so any classifier-vs-dashboard mismatch is an undercount

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_enterprise_request_metering (6d0a044) with litellm_internal_staging (4580ad0)}

[yassin-berriai]

@greptileai please review the current head 726eb1d

[yassin-berriai]

bugbot run

[yassin-berriai]

@greptileai please review the current head 5933294

[yassin-berriai]

@greptileai please review the current head 534ddfd

[yassin-berriai]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 534ddfd. Configure here.}

[veria-ai]

Medium: Replicas export conflicting cumulative counters

Every process exports this cumulative counter with the same resource attributes, meter name, and dimensions. In deployments with multiple workers or replicas, independent writers can overlap under one OTLP stream identity; receivers may deduplicate overlapping streams, so some billable requests can be dropped. Add a stable, process-unique service.instance.id resource attribute, such as an instance identifier combined with the PID or a UUID generated when the provider is built.

[yassin-berriai]

docs: BerriAI/litellm-docs#571

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[mateo-berri]

The findings are about temp file lifetime and a brief file-permission window, neither of which affects request correctness or data integrity

Is this greptile concern legit? If so, is it worth addressing?

EDIT: Ah clarified in person -> it's not important because the whole filesystem will be gone anyways when the pod goes down

[mateo-berri]

LGTM; thanks!

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit 6d0a044. Configure here.}

[cursor]

Zero export interval accepted

Medium Severity

The _export_interval_ms function allows zero or negative values for LITELLM_BILLING_METRICS_EXPORT_INTERVAL_MS. This can cause the PeriodicExportingMetricReader to export metrics continuously or excessively, leading to unnecessary CPU contention and system load.

 

^{Reviewed by Cursor Bugbot for commit 6d0a044. Configure here.}