fix(proxy): attribute provider and model_info on pre_call_hook rejections

[devin-ai-integration]

TLDR

Problem this solves:

• Requests rejected in pre_call_hook land on the spend log with no provider
• custom_llm_provider and model_id are empty on 429 and 403 failure rows
• Per-provider reporting cannot attribute rate limit or guardrail rejections

How it solves it:

• Resolve the requested model group through the router on proxy-only failures
• Stamp custom_llm_provider when every deployment in the group shares one provider
• Stamp model_info and deployment into logging metadata when the group has one deployment
• Keep those rejections out of router cooldown, RPM tracking, and Prometheus deployment state

User Flow

Before: a team whose key is limited to 1 request per minute sees the rejected second request logged with no provider

1. They send POST https://litellm-domain/v1/chat/completions with "model": "gpt-5.5" and get a 200
2. They send the same request again inside the minute and get a 429 Rate limit exceeded for api_key
3. They open GET https://litellm-domain/spend/logs?api_key= and the 200 row shows custom_llm_provider: "openai" and a model_id
4. The 429 row on the same page shows custom_llm_provider: "" and model_id: "", so their provider report cannot count it

After: the rejected request carries the same provider and deployment id as the served one

1. They send POST https://litellm-domain/v1/chat/completions with "model": "gpt-5.5" and get a 200
2. They send the same request again inside the minute and get a 429 Rate limit exceeded for api_key
3. They open GET https://litellm-domain/spend/logs?api_key= and the 200 row shows custom_llm_provider: "openai" and a model_id
4. The 429 row now shows custom_llm_provider: "openai" and the same model_id, so the provider report counts it
5. A group that mixes OpenAI and Anthropic deployments still logs the 429 row with no provider, since none can be claimed

Relevant issues

Requested by a customer (Pylon #8563)

Affected release

Linear ticket

Resolves LIT-7701

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Setup: two proxies on the same Postgres database, the merge base 25af172 on :20702 and the PR tip d6bc323 on :20701, both started with --config config.yaml --num_workers 1 --use_prisma_db_push where config.yaml sets general_settings.master_key: sk-1234, store_model_in_db: true, proxy_batch_write_at: 5. Real OpenAI and Anthropic keys, no mocks. <key> is the virtual key returned by /key/generate

Video of the Admin UI Logs page (/ui/?page=logs, request drawer) for the same two runs, Before at 25af172 and After at d6bc323 (normal 429 and caller metadata.model_info 429 legs): https://berriaillm.slack.com/archives/C0B302ZJU05/p1790101309143669?thread_ts=1790069838.026499&cid=C0B302ZJU05

Before (25af172)

single deployment, rpm_limit=1 key, second call rejected in pre_call_hook

1. curl -X POST http://localhost:20702/model/new -H "Authorization: Bearer sk-1234" -d '{"model_name":"precall-before-1790065409","litellm_params":{"model":"openai/gpt-5.5","api_key":"os.environ/OPENAI_API_KEY"}}' returns model_id=81ffcace-18f8-40a5-8563-a6598c428010
2. curl -X POST http://localhost:20702/key/generate -H "Authorization: Bearer sk-1234" -d '{"models":["precall-before-1790065409"],"rpm_limit":1}'
3. curl -X POST http://localhost:20702/chat/completions -H 'Authorization: Bearer <key>' -d '{"model":"precall-before-1790065409","messages":[{"role":"user","content":"say ok 1"}],"max_tokens":5}' returns 200 with "id":"chatcmpl-EQq4i8uXGyYye1uXcZkYdYeAbrKn3"
4. The same call with say ok 2 returns HTTP 429 {"error":{"message":"Rate limit exceeded for api_key: b0f912f0... Limit type: requests. Current limit: 1, Remaining: 0 ...","type":"throttling_error","code":"429"}}
5. curl "http://localhost:20702/spend/logs?api_key=<key>" -H "Authorization: Bearer sk-1234" after 12s returns
   {'status': 'success', 'model': 'openai/gpt-5.5', 'custom_llm_provider': 'openai', 'model_id': '81ffcace-18f8-40a5-8563-a6598c428010', 'spend': 0.0002}
{'status': 'failure', 'model': 'precall-before-1790065409', 'custom_llm_provider': '', 'model_id': '', 'spend': 0.0}
6. http://localhost:20702/ui/?page=logs, click the 429 row: the drawer shows Provider: - and Model ID: -

mixed provider group (openai + anthropic behind one name), rejected call must not claim a provider

1. Two POST /model/new calls register mixed-before-1790065409 as openai/gpt-5.5 and anthropic/claude-sonnet-4-5, then /key/generate with rpm_limit: 1
2. First /chat/completions returns 200 "id":"chatcmpl-654cc3ca-2b04-40ff-aaa8-6e28385c443a" (served by the Anthropic deployment), second returns HTTP 429 Rate limit exceeded for api_key: 36c608b3...
3. GET /spend/logs?api_key=<key> returns
   {'status': 'success', 'model': 'anthropic/claude-sonnet-4-5', 'custom_llm_provider': 'anthropic', 'model_id': '11103f23-8176-4f7b-8153-38cc5af6ff32', 'spend': 0.000108}
{'status': 'failure', 'model': 'mixed-before-1790065409', 'custom_llm_provider': '', 'model_id': '', 'spend': 0.0}

After (d6bc323)

single deployment, rpm_limit=1 key, second call rejected in pre_call_hook

1. curl -X POST http://localhost:20701/model/new -H "Authorization: Bearer sk-1234" -d '{"model_name":"db-w1-head-1790096777","litellm_params":{"model":"openai/gpt-5.5","api_key":"os.environ/OPENAI_API_KEY"}}' returns model_id=464a21e8-3ca8-4c5e-b5c3-8d6c082808b7
2. curl -X POST http://localhost:20701/key/generate -H "Authorization: Bearer sk-1234" -d '{"models":["db-w1-head-1790096777"],"rpm_limit":1}'
3. curl -X POST http://localhost:20701/chat/completions -H 'Authorization: Bearer <key>' -d '{"model":"db-w1-head-1790096777","messages":[{"role":"user","content":"say ok 1"}],"max_tokens":5}' returns 200
4. The same call with say ok 2 returns HTTP 429 Rate limit exceeded for api_key ... Limit type: requests. Current limit: 1, Remaining: 0
5. curl "http://localhost:20701/spend/logs?api_key=<key>" -H "Authorization: Bearer sk-1234" after 12s returns
   {'status': 'success', 'model': 'openai/gpt-5.5', 'custom_llm_provider': 'openai', 'model_id': '464a21e8-3ca8-4c5e-b5c3-8d6c082808b7'}
{'status': 'failure', 'model': 'openai/gpt-5.5', 'custom_llm_provider': 'openai', 'model_id': '464a21e8-3ca8-4c5e-b5c3-8d6c082808b7'}
6. http://localhost:20701/ui/?page=logs, click the 429 row: the drawer shows Provider: openai and Model ID: 464a21e8-...

caller-supplied metadata.model_info on a key with allow_client_pricing_override, second call rejected before routing

curl -X POST http://localhost:<port>/chat/completions -H 'Authorization: Bearer <key rpm_limit=1>' -d '{"model":"gpt-5.5","messages":[{"role":"user","content":"say ok"}],"max_tokens":5,"metadata":{"model_info":{"id":"spoofed-deployment-1790096752"}}}' twice returns 200 then 429 on both sides. At the merge base the 429 spend row is {'custom_llm_provider': '', 'model_id': 'spoofed-deployment-1790096752'} and /metrics counts litellm_deployment_failure_responses_total{api_provider="None",model_id="spoofed-deployment-1790096752"}, so the caller's id is charged to deployment health. At the tip the row is {'custom_llm_provider': 'openai', 'model_id': 'spoofed-deployment-1790096752'} while litellm_deployment_failure_responses_total and litellm_deployment_total_requests_total for that key have api_provider="" and model_id="": the reject is attributed in spend logs but never counted as a selected deployment

mixed provider group (openai + anthropic behind one name), rejected call must not claim a provider

1. Two POST /model/new calls register mixed-after-1790068533 as openai/gpt-5.5 and anthropic/claude-sonnet-4-5, then /key/generate with rpm_limit: 1
2. First /chat/completions returns 200 "id":"chatcmpl-EQqt92evh7pC74qoObg685p7SmDyW" (served by the OpenAI deployment), second returns HTTP 429 Rate limit exceeded for api_key: f5c868e9...
3. GET /spend/logs?api_key=<key> returns
   {'status': 'success', 'model': 'openai/gpt-5.5', 'custom_llm_provider': 'openai', 'model_id': 'c014289b-dee5-4680-82bd-d56795d0d043', 'spend': 0.0002}
{'status': 'failure', 'model': 'mixed-after-1790068533', 'custom_llm_provider': '', 'model_id': '', 'spend': 0.0}

guardrails on a single deployment guarded (openai/gpt-5.5), Prometheus enabled on both sides

A generic_guardrail_api guardrail points at a local API that always answers BLOCKED. generic-block runs pre-call, generic-block-post runs post-call after the real OpenAI response

curl -X POST http://localhost:<port>/chat/completions -H 'Authorization: Bearer <key>' -d '{"model":"guarded","messages":[{"role":"user","content":"hello"}],"guardrails":["generic-block"]}' returns HTTP 400 policy: blocked by generic guardrail api on both sides. Spend row at the merge base: {'status': 'failure', 'model': 'guarded', 'custom_llm_provider': '', 'model_id': ''}; at the tip: {'status': 'failure', 'model': 'openai/gpt-5.5', 'custom_llm_provider': 'openai', 'model_id': '767c6383...'}. On both sides litellm_deployment_failure_responses_total and litellm_deployment_total_requests_total for that key have api_provider="" and model_id="", so the pre-call block still does not count against the deployment

The same call with "guardrails":["generic-block-post"] and "max_tokens":40 returns HTTP 400 on both sides, and on both sides the spend row is {'status': 'failure', 'model': 'openai/gpt-5.5', 'custom_llm_provider': 'openai', 'model_id': '767c6383...'} and litellm_deployment_failure_responses_total keeps api_provider="openai", litellm_model_name="guarded", model_id="767c6383...", so a post-call guardrail failure stays attributed to the deployment that served it

malformed request model (Prometheus callback enabled on both sides)

curl -X POST http://localhost:<port>/chat/completions -H 'Authorization: Bearer <key>' -d '{"model":["gpt-5.5"],"messages":[{"role":"user","content":"hi"}]}' returns HTTP 400 'model' must be a string. on both sides. At the merge base the proxy log also records TypeError: unhashable type: 'list' from the Prometheus failure hook (2 occurrences per run); at the tip the log has none and /metrics counts the request under requested_model="other"

Same scenarios were also run with --num_workers 2 and --num_workers 4 on both sides (8 or more requests per scenario, 4 server processes confirmed per side at 4 workers) with identical row shapes, and under 40 concurrent rejected requests with hostile metadata bodies and a 20s Postgres outage, where both sides kept returning 429 and recovered

team public model name that collides with a global group, Prometheus and generic_api sink enabled on both sides

Config group shared-name is one anthropic/claude-sonnet-4-5 deployment. POST /model/new adds a team-only openai/gpt-5.5 deployment with team_public_model_name: shared-name, then /key/generate with that team and rpm_limit: 1. Two calls to /chat/completions with "model":"shared-name" return 200 then HTTP 429 Rate limit exceeded for api_key: 4ac97abb...

GET /spend/logs?api_key=<key> at the merge base: {'status': 'failure', 'model': 'shared-name', 'custom_llm_provider': '', 'model_id': ''}; at the tip: {'status': 'failure', 'model': 'openai/gpt-5.5', 'custom_llm_provider': 'openai', 'model_id': 'team-openai-dep-head-1790111360'}, the team deployment and not the global anthropic one. /metrics at the merge base labels that 429 litellm_proxy_failed_requests_metric_total{api_provider="anthropic",model_id="None",requested_model="shared-name"}, the global group leaking into a team key's reject; at the tip it is {api_provider="openai",model_id="team-openai-dep-head-1790111360",requested_model="shared-name"}

Prometheus deployment state A/B on the single deployment group solo (openai/gpt-5.5), key with rpm_limit: 1, 200 then 429

On both sides litellm_deployment_state{model_id="2415f9c0..."} stays 0.0 and litellm_deployment_failure_responses_total / litellm_deployment_total_requests_total for the reject keep api_provider="", model_id="". At the merge base litellm_proxy_failed_requests_metric_total{api_provider="openai",model_id="None",requested_model="solo"} and litellm_llm_api_failed_requests_metric_total{model="solo",model_id=""}; at the tip both carry model_id="2415f9c0..."

non-Postgres sink: generic_api callback posting to a local HTTP receiver

Batch payloads for the two 429 rows above at the merge base: status=failure custom_llm_provider=None model_id=''; at the tip: status=failure custom_llm_provider=openai model_id=team-openai-dep-head-1790111360 and model_id=2415f9c0.... Success rows identical on both sides

Type

🐛 Bug Fix

Caveats (if any)

Medium

• A group with one resolvable and one unresolvable deployment gets no provider, by design
• Only metadata.model_info and metadata.deployment are stamped, never top-level litellm_params.model_info, so router cooldown and RPM tracking ignore these rejections
• Failure logs for pre-routing rejects carry litellm_params.proxy_rejected_before_routing, which Prometheus uses to clear deployment labels and skip deployment state updates. The flag keys off the logging object's first_api_call_start_time (set at provider handoff), not on metadata contents, so caller-supplied metadata.model_info cannot suppress it and post-call failures never get it

Low

• model_group is blank on failure payloads sent to the generic_api sink on both the merge base and the tip, unrelated to this change
• The code-quality CI job fails on litellm/proxy/bug_report_config.py::_render_json, which landed on main in a9cea9d and fails identically on main
• Model names not in the router (unknown model or no router) keep today's behavior

QA runbook

• tests/e2e/quota_management/spend_tracking/test_spend_tracking_e2e.py::test_pre_call_rejection_row_attributes_provider_and_model_id - a request rejected by the key rate limit before routing still lands its single deployment's provider and model id on the failure spend row
  • POST /model/new with the master key registering openai/gpt-5.5 under a fresh name with api_key: os.environ/OPENAI_API_KEY (needs STORE_MODEL_IN_DB=True and a real OPENAI_API_KEY), note the returned model_id
  • POST /key/generate with {"models":["<that name>"],"rpm_limit":1}
  • Send POST /v1/chat/completions with that key and max_tokens: 8, expect 200
  • Send it again inside the same minute, expect 429 naming the request limit
  • GET /spend/logs?api_key= until a status: failure row appears (batch writes flush every few seconds)
  • Expect the failure row's custom_llm_provider to equal the success row's (openai) and its model_id to equal the id from step 1
  • Sanity check: this test makes sense to add and is not hand-wavey (e.g., assert actual expected spend instead of just spend > 0) or potentially flaky

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

---

Note

Medium Risk
Touches failure logging, spend attribution, and Prometheus deployment metrics on hot proxy paths; behavior is guarded by router resolution rules and explicit proxy-rejection detection.

Overview
Pre-call proxy rejections (rate limits, guardrails, auth-style HTTPException / ProxyException) now get provider and deployment attribution on failure spend logs when the requested model group unambiguously maps through the router, without treating those rejects as deployment failures.

Proxy logging resolves the model group (respecting team-scoped deployments) and stamps custom_llm_provider when all deployments share one provider, plus metadata.model_info / deployment only for single-deployment groups—deliberately not top-level model_info so router cooldown and per-deployment RPM counters are unchanged.

Prometheus treats the same proxy-side exception types as “no deployment picked” for deployment health metrics even when logs carry a model_id, and collapses non-string model values on proxy request metrics to the other label to avoid cardinality / hook crashes.

E2E and unit tests cover attributed failure rows, mixed-provider omission, and deployment-state behavior.

^{Reviewed by Cursor Bugbot for commit fc4f4cd. Bugbot is set up for automated code reviews on this repo. Configure here.}

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ yucheng-berri
❌ devin-ai-integration[bot]
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_pre_call_reject_provider_attribution (d6bc323) with main (ded69f8)¹}

Footnotes

1. No successful run was found on main (9e92fad) during the generation of this report, so ded69f8 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[greptile-apps]

Confidence Score: 5/5

The PR appears safe to merge; no actionable regression or outstanding previous finding remains.

Summary

This PR improves failure attribution for requests rejected before provider routing.

• Resolves unambiguous router model groups to populate provider and single-deployment metadata in failure spend logs.
• Uses a proxy-owned pre-routing flag so attributed rejections do not affect deployment health, cooldown, or RPM metrics.
• Bounds malformed non-string requested-model metric labels to the existing other bucket.
• Adds unit and E2E coverage for single, mixed, unresolved, team-scoped, spoofed-metadata, and post-routing cases.

_{Reviews (9) · Last reviewed commit: "test(e2e): poll for both served and reje..."}

[codecov]

Codecov Report

❌ Patch coverage is 96.07843% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/integrations/prometheus.py	85.71%	1 Missing ⚠️
litellm/proxy/utils.py	97.67%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[devin-ai-integration]

@greptileai

[devin-ai-integration]

@greptileai

[devin-ai-integration]

@greptileai

[devin-ai-integration]

@greptileai

[cursor]

Stale Bugbot comment from a previous run.

[mateo-berri]

bugbot run

[devin-ai-integration]

@greptileai

[cursor]

Stale Bugbot comment from a previous run.

[devin-ai-integration]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[mateo-berri]

bugbot run

[devin-ai-integration]

@greptileai

[cursor]

Stale Bugbot comment from a previous run.

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit d6bc323. Configure here.}