fix(rate-limits): keep the v3 limiter out of provider-facing metadata on responses routes

[devin-ai-integration]

TLDR

Problem this solves:

• Rate limits made /v1/responses send metadata upstream
• Backends rejecting Responses metadata returned HTTP 400
• Internal stash keys landed in caller-supplied provider metadata

How it solves it:

• Limiter writes its stash to one internal bucket
• That bucket is litellm_metadata whenever the route owns it
• Provider-facing metadata is never created or touched

Relevant issues

Fixes #35197

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Proof of fix posted as a PR comment: live proxy on localhost:4000, real provider calls, before/after commit hashes and the raw upstream request body captured from --detailed_debug

Type

🐛 Bug Fix

Changes

_PROXY_MaxParallelRequestsHandler stashed every rate-limit value into both metadata channels at once:

for channel in ("metadata", "litellm_metadata"):
    existing = data.get(channel)
    if isinstance(existing, dict):
        existing[key] = value
    elif channel == "metadata":
        data[channel] = {key: value}

For chat completions that is harmless, since metadata is in all_litellm_params and never reaches the provider. For the routes in LITELLM_METADATA_ROUTES (Responses, /v1/messages, batches, files) litellm_metadata is the internal channel and metadata is a real provider request parameter, so the branch above conjured a provider-visible metadata object out of an internal bookkeeping write; OpenAI-compatible backends that don't implement Responses metadata answered 400. #32711 made this fire on every successful rate-limit check, so any key/user/team/project/org/agent RPM, TPM, or max-parallel limit was enough to trigger it

The write now goes through the existing get_or_create_metadata_bucket helper, which resolves the single proxy-internal bucket the same way every other reader and writer of internal metadata does, so a Responses request only ever grows litellm_metadata, and a caller that supplies its own metadata gets it back untouched

_lookup_stashed_value also learned to read litellm_params["litellm_metadata"], not just litellm_params["metadata"]; with the dual write gone, that is the channel the value lives in when a Responses caller supplied provider metadata of their own

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/09dfb72721944792b432e6de150873fd

---

Note

Medium Risk
Changes the pre-call rate limiter’s request-data mutation on a hot proxy path; scope is narrow (metadata channel selection) but incorrect bucket choice could break TPM reconciliation or logging on mixed routes.

Overview
Fixes #35197: successful rate-limit checks on Responses-style routes were writing internal stash keys into metadata, which is forwarded to the provider and caused HTTP 400 from backends that reject unknown Responses metadata.

The v3 parallel request limiter no longer dual-writes to metadata and litellm_metadata. Stash values (TPM reservation, rate-limit descriptors, parallel slot markers, etc.) go only through get_or_create_metadata_bucket, so internal state lands in litellm_metadata on routes that own it and never creates or mutates caller metadata.

_lookup_stashed_value now resolves stash keys from litellm_params["litellm_metadata"] as well as litellm_params["metadata"], matching the single-bucket write path when callers supply their own provider metadata.

A regression test covers aresponses pre-call behavior with and without caller-supplied metadata.

^{Reviewed by Cursor Bugbot for commit 68de15b. Bugbot is set up for automated code reviews on this repo. Configure here.}

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ yucheng-berri
❌ devin-ai-integration[bot]
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

The PR keeps rate-limiter state in the route-appropriate internal metadata bucket.

• Replaces dual-channel stash writes with get_or_create_metadata_bucket.
• Adds lookup support for litellm_params["litellm_metadata"].
• Adds Responses-route coverage confirming provider metadata remains untouched and directly exercises the new internal lookup channel.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the prior test-coverage concern is resolved because the updated assertion passes the stash through litellm_params["litellm_metadata"] and therefore directly exercises the newly added lookup branch.

Important Files Changed

Filename	Overview
litellm/proxy/hooks/parallel_request_limiter_v3.py	Routes limiter stash writes through the internal metadata helper and reads the route-owned metadata channel without exposing a new actionable defect.
tests/test_litellm/proxy/hooks/test_parallel_request_limiter_v3.py	Adds regression coverage for absent and caller-supplied provider metadata and now directly exercises lookup through litellm_params["litellm_metadata"].

_{Reviews (2): Last reviewed commit: "Update tests/test_litellm/proxy/hooks/te..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 93.33333% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/proxy/hooks/parallel_request_limiter_v3.py	93.33%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_fix_35197_responses_metadata_leak (f485735) with litellm_internal_staging (4eecf7a)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (5c16132) during the generation of this report, so 4eecf7a was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[yucheng-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 68de15b. Configure here.}

[mateo-berri]

LGTM

Pretty bad pattern to have internal bookkeeping. I think it was really hard to understand the fact that we pop for security reasons, then we push the bookkeeping params, then we pop again with all_litellm_params and we use metadata for chat completions and litellm_metadata for responses (because responses standard has metadata). I am reworking this to be typed async safe contextvar here: #35278

[yucheng-berri]

@greptileai review latest head