fix(bedrock): close anthropic passthrough metadata to provider schema at egress - #31017
Draft
mateo-berri wants to merge 2 commits into
Draft
Conversation
… at egress Anthropic-on-Bedrock accepts metadata.user_id and rejects every other metadata field with "metadata.<field>: Extra inputs are not permitted". The proxy enriches the request metadata with internal spend fields (tags, spend_logs_metadata, user_api_key_*) and the invoke/converse passthrough route forwards that body verbatim, so the enrichment reaches AWS and 400s the request (#30629) Project the forwarded metadata down to the provider's closed schema (AnthropicMetadata, user_id only) in BedrockPassthroughConfig.sign_request, right before the body is signed. This is a single typed egress choke point that drops any internal field instead of chasing each enrichment site route by route, which is the structural hardening #30301 asks for. A new body is built rather than the cached body mutated, so spend tracking still sees the stripped fields
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Relevant issues
Implements the egress hardening from the RCA in #30999 and the category-level guard tracked in #30301, for the leak reported in #30629. Complements #30985 (merged), which stops key/team/header spend tags from landing in
metadataat the source; this PR adds the structural guard at the single provider egress so any other internal field that reachesmetadatais dropped too.Linear ticket
N/A
Pre-Submission checklist
make test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewScreenshots / Proof of Fix
No mock; run against a live proxy hitting real AWS Bedrock. Run the proxy with
python litellm/proxy/proxy_cli.py --config litellm/proxy/dev_config.yaml --detailed_debug --reload 2>&1 | tee litellm.logand a Bedrock invoke config modelbedrock-invoke-sonnet-4-6pointing atus.anthropic.claude-sonnet-4-6.metadatawithuser_idplus a non-user_idfield. Before this PR AWS returnsHTTP 400 metadata.tags: Extra inputs are not permitted; with this PR the field is stripped at the egress and the request returnsHTTP 200, withuser_idstill forwarded:GET /spend/logsfor that request still showsuser_abc123, so closing the provider schema does not drop billing data.Type
🐛 Bug Fix
Changes
Anthropic-on-Bedrock accepts
metadata.user_idand rejects every other metadata field withmetadata.<field>: Extra inputs are not permitted. The proxy enriches the request metadata with LiteLLM-internal fields (spend tags,spend_logs_metadata,user_api_key_*), and the invoke/converse passthrough route forwards the request body verbatim, so that enrichment reaches AWS and 400s the request.#30985 fixed the key/team/header tag vector at the source by routing tags into
litellm_metadatafor Bedrock routes. That is a per-route source patch, so it does not cover the rest of the category: any internal field that still lands inmetadata, or a client-supplied non-user_idfield, leaks again. This PR closes the category at the one place every invoke/converse passthrough body is serialized to AWS.restrict_anthropic_bedrock_metadataprojects the forwarded body'smetadatadown to the provider's closed schema (the existingAnthropicMetadata,user_idonly) insideBedrockPassthroughConfig.sign_request, right before the body is signed. It is gated onanthropic_versionso only Anthropic-native invoke bodies are touched; Titan/Cohere/Llama/Nova bodies and the Converse schema are forwarded unchanged. The projection builds a new body rather than mutating the cached one, so spend tracking still reads the tags it needs for billing while the bytes signed and sent to AWS carry onlyuser_id. This mirrors the closed-schema strip already shipped for/chat/completions(#24661) and/v1/messages(validate_anthropic_api_metadata), now applied at the Bedrock passthrough egress.Files:
litellm/llms/bedrock/passthrough/transformation.py: addrestrict_anthropic_bedrock_metadataand call it inBedrockPassthroughConfig.sign_requesttests/test_litellm/llms/bedrock/passthrough/test_bedrock_passthrough_transformation.py: regression tests for stripping every non-user_idfield, preservinguser_id, not mutating the input body, theanthropic_version/no-metadatano-op guards, and a wiring test that fails ifsign_requeststops feeding the projected body to the signer