fix(proxy): stop x-litellm-tags header tags leaking into Bedrock passthrough body

[mateo-berri]

Relevant issues

Follow-up to #30629. That issue and its fix (#30985) cover key-level metadata.tags leaking into the Bedrock passthrough body. This PR closes the sibling vector: tags supplied via the x-litellm-tags request header.

Linear ticket

N/A

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

apply_client_tag_policy_pre_auth merges x-litellm-tags into the request body's metadata dict before auth so _tag_max_budget_check can see them. It did this with an in-place write. That metadata dict is shared by reference with the cached parsed body (request.scope["parsed_body"]), and the Bedrock passthrough route re-reads that same cached body and forwards it verbatim to AWS. So the LiteLLM-internal billing tags ended up under metadata in the upstream payload, and Bedrock's Anthropic schema rejects any non-user_id key there with HTTP 400 "Extra inputs are not permitted".

The fix builds a new metadata dict and reassigns request_data[metadata_var] rather than mutating the shared object. The budget check still sees the merged tags on request_data; the cached body the route forwards stays clean.

To reproduce against a live proxy (real Bedrock call, costs real money), with a Bedrock passthrough model configured and the proxy running on localhost:4000:

curl -sS -i http://localhost:4000/bedrock/model/us.anthropic.claude-sonnet-4-20250514-v1:0/invoke \
  -H "Authorization: Bearer $LITELLM_KEY" \
  -H "Content-Type: application/json" \
  -H "x-litellm-tags: team-a,cost-center-7" \
  -d '{
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 16,
        "messages": [{"role": "user", "content": "ping"}],
        "metadata": {"user_id": "end-user-1"}
      }'

Before this change the call returns HTTP 400 with a validation error on metadata.tags; after it returns HTTP 200, and the team-a / cost-center-7 tags are still recorded for spend tracking (visible at http://localhost:4000/ui/?page=logs).

Type

🐛 Bug Fix

Changes

litellm/proxy/litellm_pre_call_utils.py: apply_client_tag_policy_pre_auth now constructs a fresh metadata dict instead of mutating the request body's metadata object in place, so header tags can no longer leak through the shared cached body into a passthrough provider payload.

tests/test_litellm/proxy/test_litellm_pre_call_utils.py: regression test driving the merge through the real request-body cache and asserting the re-read (forwarded) body carries no tags while the auth-time body still exposes them to _tag_max_budget_check. It fails before the fix and passes after.

[mateo-berri]

Superseded by the latest commit on #30985 (dbd1600), which pre-seeds litellm_metadata before apply_client_tag_policy_pre_auth. For Bedrock and the other LITELLM_METADATA_ROUTES that routes header-supplied x-litellm-tags into litellm_metadata instead of the provider-facing metadata field, which closes the same header-tag leak this PR targeted, and it does so as part of the complete #30629 fix covering both the key-tag and header-tag vectors. Closing this as redundant.

One thing worth keeping in mind as a possible follow-up: that pre-seed is scoped to LITELLM_METADATA_ROUTES, so a passthrough provider not on that list would still forward header tags in its body. If that ever becomes a real case, hardening apply_client_tag_policy_pre_auth to never mutate the shared cached request body dict would close it generally

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

This PR attempts to fix x-litellm-tags header values leaking into the Bedrock passthrough body by building a fresh metadata dict in apply_client_tag_policy_pre_auth instead of mutating the existing one in-place.

• Core fix (litellm_pre_call_utils.py): Replaces metadata["tags"] = … (in-place mutation) with request_data[key] = {**existing_metadata, "tags": …} (new dict assignment) to avoid modifying the shared cached body.
• New test (test_litellm_pre_call_utils.py): Verifies that after the merge, the re-read forwarded body is clean by pre-populating request.scope["parsed_body"] before the first _read_request_body call.

Confidence Score: 2/5

The fix avoids in-place mutation of the shared metadata dict, but the reassignment still propagates into the cached parsed body when auth is the first reader — which is the standard request flow — leaving tags visible in the body forwarded to Bedrock.

The new dict assignment works only when request_data is a shallow copy returned from a cache hit (pre-cached body). In the actual passthrough flow, user_api_key_auth reads the body first with an empty cache, receiving parsed_body directly. The assignment request_data["metadata"] = new_dict_with_tags then also sets parsed_body["metadata"] = new_dict_with_tags, and the route handler's subsequent _read_request_body call reconstructs the dict from that same parsed_body — so the tags are still present in the forwarded body. The regression test passes only because it artificially pre-caches the body before calling _read_request_body, putting it into a different code path than production. The underlying leak is not closed for the standard Bedrock passthrough flow.

Both changed files need attention: litellm/proxy/litellm_pre_call_utils.py for the incomplete fix, and tests/test_litellm/proxy/test_litellm_pre_call_utils.py for the test that does not exercise the production code path.

Important Files Changed

Filename	Overview
litellm/proxy/litellm_pre_call_utils.py	apply_client_tag_policy_pre_auth now builds a new metadata dict instead of mutating in-place, but the assignment request_data[key] = new_dict still modifies parsed_body when auth is the first caller of _read_request_body (the production case), so tags can still leak into the forwarded Bedrock body.
tests/test_litellm/proxy/test_litellm_pre_call_utils.py	New regression test pre-caches the parsed body via _safe_set_request_parsed_body before calling _read_request_body, which puts the test into a different code path (cache-hit returns a shallow copy) than the production flow (first-ever call returns parsed_body directly). The test passes but does not verify the actual production scenario.

_{Reviews (1): Last reviewed commit: "fix(proxy): stop x-litellm-tags header t..." | Re-trigger Greptile}

[greptile-apps]

Test pre-caches the body, masking the production failure path

The test calls _safe_set_request_parsed_body before _read_request_body, so _read_request_body hits the cache path and returns a new shallow-copy dict ({key: parsed_body[key] for key in accepted_keys}). This means auth_body IS NOT original_body, so assigning request_data["metadata"] = new_dict only updates the shallow copy — original_body["metadata"] is untouched and the second _read_request_body call returns a clean body.

In the actual Bedrock passthrough production flow, user_api_key_auth (a FastAPI Depends) is the first caller of _read_request_body. Because no cache exists yet, _read_request_body parses the body, stores (tuple(parsed_body.keys()), parsed_body) in request.scope["parsed_body"], and returns parsed_body directly — not a copy. At that point request_data IS parsed_body. When apply_client_tag_policy_pre_auth then does request_data["metadata"] = new_dict_with_tags, it also sets parsed_body["metadata"] = new_dict_with_tags. When bedrock_llm_proxy_route later calls _read_request_body, it retrieves the cache and reconstructs {key: parsed_body[key] for key in accepted_keys} — where "metadata" now maps to new_dict_with_tags. Tags still leak.

To reproduce the actual production path, the test should not pre-populate the cache; it should let _read_request_body read raw bytes on the first call (as auth does), then assert on the second read.

[greptile-apps]

New dict assignment still leaks when request_data IS parsed_body

The fix avoids mutating the existing metadata dict in-place, but the assignment request_data[_metadata_variable_name] = new_dict still changes parsed_body["metadata"] whenever request_data and parsed_body are the same object — which is the case in the standard production flow.

user_api_key_auth is the first code to call _read_request_body on a Bedrock passthrough request. Because the cache is empty at that point, _read_request_body parses the raw body, caches (tuple(keys), parsed_body), and returns parsed_body directly (not a copy). Consequently request_data IS parsed_body. The fix then does request_data["metadata"] = new_dict_with_tags, which is identical to parsed_body["metadata"] = new_dict_with_tags. When bedrock_llm_proxy_route later calls _read_request_body, _safe_get_request_parsed_body returns {key: parsed_body[key] for key in accepted_keys}, and parsed_body["metadata"] is now the new dict with tags — so they are forwarded to Bedrock unchanged.

A robust fix would need to either (a) save a snapshot of the original body in request.scope before auth-time mutations (separate from the live parsed_body entry the route handler reads), or (b) strip LiteLLM-internal keys from data["data"] in the passthrough handler before serialising and forwarding the body.