Repository navigation
fix: prevent key-level metadata.tags from leaking into Bedrock passthrough body #30985
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 3 commits
fe5164c
b98ef2a
8beba99
dbd1600
6bd0af2
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -108,6 +108,7 @@ def parse_cache_control(cache_control): | |
|
|
||
| LITELLM_METADATA_ROUTES = ( | ||
| "batches", | ||
| "bedrock", | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. why not add all pass throughs endpoints here ?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Bedrock is the only passthrough that needs to be in LITELLM_METADATA_ROUTES. Every other passthrough goes through create_pass_through_route, which already strips both metadata and litellm_metadata from the forwarded body, so tags never leak there Only Bedrock uses the body-preserving base_passthrough_process_llm_request path that leaks Adding them all would also be unsafe, since the /openai substring would catch the native /openai/deployments/.../chat/completions route and wrongly flip its metadata field They follow different paths because Bedrock uses the heavier allm_passthrough_route path (for SigV4 signing, URL building, router-model resolution, cost tracking) which signs and forwards the body verbatim, no strip |
||
| "/v1/messages", | ||
| "responses", | ||
| "files", | ||
|
|
@@ -1468,8 +1469,7 @@ async def add_litellm_data_to_request( | |
| _metadata_variable_name=_metadata_variable_name, | ||
| ) | ||
|
|
||
| # Add headers to metadata for guardrails to access (fixes #17477) | ||
| # Guardrails use metadata["headers"] to access request headers (e.g., User-Agent) | ||
| # Expose request headers under the metadata field for guardrails (fixes #17477) | ||
| if _metadata_variable_name in data and isinstance( | ||
| data[_metadata_variable_name], dict | ||
| ): | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.