Fix Bedrock passthrough call ID headers - #27412
Conversation
Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com>
|
|
Greptile SummaryThis PR fixes a header propagation bug where non-streaming Bedrock passthrough responses were losing LiteLLM metadata headers (e.g.,
Confidence Score: 5/5Safe to merge — the change is minimal and isolated to the non-streaming passthrough rebuild path, with a targeted regression test that directly validates the fix. The one-line change correctly threads the accumulated fastapi_response headers into get_response_headers, which already has safe merge semantics (provider headers first, then custom headers on top). The streaming path is unaffected. The test is mock-only, covers the exact failure mode described in the PR, and makes no real network calls. No files require special attention.
|
| Filename | Overview |
|---|---|
| litellm/proxy/common_request_processing.py | Single-line fix: passes accumulated fastapi_response headers as custom_headers when rebuilding the non-streaming passthrough Response, ensuring x-litellm-call-id and other LiteLLM metadata headers are included in the client response. |
| tests/test_litellm/proxy/test_common_request_processing.py | Adds a regression test using mocks that simulates the Bedrock non-streaming passthrough path, verifying both provider headers and LiteLLM metadata headers survive in the final response. No real network calls made. |
Reviews (1): Last reviewed commit: "Preserve LiteLLM headers for passthrough..." | Re-trigger Greptile
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
b891a20
into
litellm_internal_staging
Co-authored-by: oss-agent-shin <279349115+oss-agent-shin@users.noreply.github.com> Co-authored-by: ishaan-berri <ishaan-berri@users.noreply.github.com>
Summary
Non-streaming ALLM passthrough responses were rebuilding a FastAPI response from the upstream provider response after common request processing had already attached LiteLLM metadata headers to the injected response object. The fix merges those accumulated LiteLLM headers into the final passthrough response so Bedrock Invoke responses include the call ID while preserving provider headers.
Repro
Call a non-streaming Bedrock passthrough route such as
/bedrock/model/<bedrock-model-id>/invoke. Before the fix, provider headers likex-amzn-requestidwere returned, butx-litellm-call-idwas missing from the client response.Evidence
Goal: the final passthrough response should include the upstream Bedrock-style headers and LiteLLM metadata headers, including
x-litellm-call-id.Focused proof run after the fix:
Regression observed before the fix:
Tests
tests/test_litellm/proxy/test_common_request_processing.py::TestProxyBaseLLMRequestProcessing::test_base_passthrough_process_llm_request_preserves_litellm_headers_for_non_streaming_response.KeyError: x-litellm-call-id.uv run pytest tests/test_litellm/proxy/test_common_request_processing.py -q-> 87 passed..github/workflows/test-linting.ymlpassed locally: lock check, frozen sync, Black check, Ruff, MyPy, circular import check, and import safety.CI
LiteLLM Linting, unit-test workflows, UI build, code quality, Semgrep, and MCP tests.realtime_translation_testingis failing on the PR and also failing onlitellm_internal_staging, so it appears pre-existing.litellm_utils_testingis failing on the PR while the latest base-branch status is green; CircleCI job logs were not accessible from this environment, and the code change is limited to proxy passthrough response headers plus a proxy unit test.Review
Relevant issues
Internal bug report.
Linear ticket
N/A
Pre-Submission checklist
tests/test_litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewScreenshots / Proof of Fix
See Evidence above.
Type
🐛 Bug Fix
✅ Test
Changes