fix(anthropic_messages): forward provider response headers on streaming /v1/messages responses

[mateo-berri]

Relevant issues

Linear ticket

Part of LIT-3724 (issue 2: x-amzn-RequestId / x-amzn-trace-id not returned for Bedrock)

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Live proxy with a Bedrock invoke model (real Bedrock API, us.anthropic.claude-opus-4-1-20250805-v1:0):

model_list:
  - model_name: bedrock-claude-real
    litellm_params:
      model: bedrock/invoke/us.anthropic.claude-opus-4-1-20250805-v1:0
      aws_region_name: us-east-1
      aws_profile_name: litellm-dev

Before (proxy on litellm_internal_staging, port 4003): a streaming /v1/messages call returns zero provider headers, so there is no way to get the Bedrock request id for an AWS support case

$ curl -s -D - -o /dev/null http://127.0.0.1:4003/v1/messages -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
    -d '{"model": "bedrock-claude-real", "max_tokens": 100, "stream": true, "messages": [{"role": "user", "content": "Say hello in three words"}]}' | grep -icE "amzn|llm_provider"
0

After (proxy on this branch, port 4002): the same call surfaces Bedrock's response headers, including the request id AWS support asks for

$ curl -s -D - -o /dev/null http://127.0.0.1:4002/v1/messages -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
    -d '{"model": "bedrock-claude-real", "max_tokens": 100, "stream": true, "messages": [{"role": "user", "content": "Say hello in three words"}]}' | grep -iE "^HTTP|amzn|llm_provider"

HTTP/1.1 200 OK
llm_provider-date: Sat, 04 Jul 2026 23:32:29 GMT
llm_provider-content-type: application/vnd.amazon.eventstream
llm_provider-transfer-encoding: chunked
llm_provider-connection: keep-alive
llm_provider-x-amzn-requestid: 77fd030e-ef64-4cc3-b782-9f4cd82673f0
llm_provider-x-amzn-bedrock-content-type: application/json

QA rerun (e2e, real Bedrock)

Independent end-to-end rerun against the real Bedrock API (us.anthropic.claude-opus-4-1-20250805-v1:0, us-east-1) through a fresh venv and a fresh proxy on a random port (64843), same config and identical commands on both legs. The before leg ran with the worktree checked out at base 26c0c93dece5182921e11387253a39dd8086db6e (origin/litellm_internal_staging) and the after leg at PR head af067b10c9289124a323f8b520aac6f41004535b, restarting the proxy between legs

Before, at the base SHA, the streaming call returns no provider headers (grep -icE "amzn|llm_provider" counts 0)

$ curl -s -D - -o /dev/null http://127.0.0.1:64843/v1/messages -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
    -d '{"model": "bedrock-claude-real", "max_tokens": 100, "stream": true, "messages": [{"role": "user", "content": "Say hello in three words"}]}' | grep -iE "^HTTP|amzn|llm_provider"

HTTP/1.1 200 OK

After, at the PR head, the identical call surfaces the Bedrock headers (count 6)

$ curl -s -D - -o /dev/null http://127.0.0.1:64843/v1/messages -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
    -d '{"model": "bedrock-claude-real", "max_tokens": 100, "stream": true, "messages": [{"role": "user", "content": "Say hello in three words"}]}' | grep -iE "^HTTP|amzn|llm_provider"

HTTP/1.1 200 OK
llm_provider-date: Sun, 05 Jul 2026 00:30:21 GMT
llm_provider-content-type: application/vnd.amazon.eventstream
llm_provider-transfer-encoding: chunked
llm_provider-connection: keep-alive
llm_provider-x-amzn-requestid: ecc93c7a-db50-4dc6-b179-2b2ba5633a2b
llm_provider-x-amzn-bedrock-content-type: application/json

A non-streaming ("stream": false) call was also run on both legs and behaves identically before and after (no llm_provider-* headers on either, matching the non-streaming limitation described under Changes)

Type

🐛 Bug Fix

Changes

Streaming /v1/messages responses were returned to the proxy as bare async generators, which cannot carry _hidden_params, so the upstream provider's HTTP response headers were dropped and return_response_headers: true had no effect on this route. For Bedrock that meant x-amzn-RequestId / x-amzn-trace-id were never surfaced, which customers need to open AWS support cases about streaming failures

async_anthropic_messages_handler now wraps the streaming return (both the direct stream and the agentic iterator) in AnthropicMessagesStreamingResponse, a thin async-iterator wrapper that carries _hidden_params["additional_headers"] built from the upstream httpx response headers via the existing process_response_headers helper (same mechanism CustomStreamWrapper and the google_genai streaming path use). The proxy's existing header plumbing in base_process_llm_request picks these up unchanged and emits them as llm_provider-* response headers on the SSE response for every provider on this route (Bedrock x-amzn-*, Anthropic request-id, etc)

Non-streaming /v1/messages responses are a TypedDict that cannot carry _hidden_params at all (a known limitation, see _response_cost_from_logging_obj); forwarding headers there needs proxy-side plumbing changes and is intentionally left out of this PR's scope

The wrapper also forwards aclose to the wrapped stream, and AgenticAnthropicStreamingIterator closes its inner and follow-up streams, so the proxy's streaming cleanup (the hasattr(response, "aclose") check in _finalize_streaming_generator_cleanup) still releases the upstream provider connection on client disconnect instead of leaving it to garbage collection

The added tests drive async_anthropic_messages_handler with an injected mock HTTP client whose response carries x-amzn-requestid / x-amzn-trace-id, for both the direct streaming branch and the agentic branch (a callback overriding async_should_run_agentic_loop), and assert the returned stream satisfies the async-iterator protocol the proxy detects, exposes the llm_provider-* prefixed headers in _hidden_params["additional_headers"], and still yields the SSE bytes unchanged. Two aclose regression tests assert that closing the wrapper closes the upstream generator on both branches

---

Note

Medium Risk
Touches the proxy-facing streaming path for all /v1/messages providers; behavior change is additive (headers + cleanup) with regression tests, but mis-wrapped streams could still affect disconnect handling.

Overview
Fixes LIT-3724 where streaming /v1/messages dropped upstream provider HTTP headers (e.g. Bedrock x-amzn-requestid) because the handler returned bare async generators with no _hidden_params.

async_anthropic_messages_handler now wraps both the direct SSE stream and the agentic AgenticAnthropicStreamingIterator in AnthropicMessagesStreamingResponse, which keeps _hidden_params["additional_headers"] (via process_response_headers, same pattern as other streaming paths) so the proxy can emit llm_provider-* headers on the SSE response. SSE bytes are unchanged.

The wrapper and agentic iterator implement aclose (via aclose_if_supported) so proxy streaming cleanup still tears down the upstream connection on client disconnect. Non-streaming header forwarding is out of scope.

^{Reviewed by Cursor Bugbot for commit af067b1. Bugbot is set up for automated code reviews on this repo. Configure here.}

Summary by CodeRabbit

• New Features

  • Streaming responses now preserve provider response headers for passthrough use.
  • Improved handling of streamed Anthropic Messages responses so header context stays available during iteration.

• Bug Fixes

  • Fixed a regression where important upstream headers could be lost when using streamed message responses.
  • Ensured streamed content still arrives normally while metadata is retained.

[mateo-berri]

@greptileai

[greptile-apps]

Greptile Summary

Streaming /v1/messages responses were returned as bare async generators that cannot carry attributes, so upstream provider HTTP headers (e.g. Bedrock's x-amzn-requestid) were silently dropped and never forwarded to clients. The fix wraps both the direct and agentic streaming branches in AnthropicMessagesStreamingResponse, a thin async-iterator class that attaches _hidden_params["additional_headers"] built from the upstream httpx response via the existing process_response_headers helper — the same mechanism already used by CustomStreamWrapper.

• AnthropicMessagesStreamingResponse is introduced in streaming_iterator.py; it delegates iteration and aclose to the wrapped stream and exposes _hidden_params["additional_headers"] for the proxy's existing header-plumbing in base_process_llm_request.
• AgenticAnthropicStreamingIterator gains an aclose method (via the new aclose_if_supported / SupportsAclose protocol helpers) so client-disconnect cleanup correctly releases both the inner stream and any follow-up iterator.
• Four new mock-only tests cover: direct streaming headers, agentic streaming headers, and aclose delegation on both paths.

Confidence Score: 5/5

Safe to merge — change is narrowly scoped to wrapping the streaming return value without touching request logic, auth, billing, or any non-streaming path.

The wrapper is a thin delegation layer over an already-working async iterator; headers are captured once from the completed httpx response before iteration starts, so no race conditions. The aclose delegation chain is correct for both the direct and agentic branches. Tests use mocks only (no real network calls) and cover all four relevant scenarios. No changes to auth, cost tracking, or request transformation.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/llms/anthropic/experimental_pass_through/messages/streaming_iterator.py	Adds AnthropicMessagesStreamingResponse wrapper class, SupportsAclose protocol, aclose_if_supported helper, and anthropic_messages_stream_hidden_params; all logic is straightforward and delegation is correct
litellm/llms/anthropic/experimental_pass_through/messages/agentic_streaming_iterator.py	Adds aclose method to AgenticAnthropicStreamingIterator that properly closes both inner and follow-up iterators via aclose_if_supported
litellm/llms/custom_httpx/llm_http_handler.py	Both streaming return sites now wrap in AnthropicMessagesStreamingResponse with headers captured once from response.headers before iteration begins; non-streaming path is correctly left unchanged
tests/test_litellm/llms/custom_httpx/test_llm_http_handler.py	Four new mock-only tests cover direct streaming headers, agentic streaming headers, and aclose delegation on both paths; no real network calls

_{Reviews (3): Last reviewed commit: "fix(anthropic_messages): forward aclose ..." | Re-trigger Greptile}

[greptile-apps]

Greptile Summary

This PR fixes streaming /v1/messages responses dropping upstream provider HTTP headers (e.g., Bedrock's x-amzn-requestid) by wrapping the bare async generator in a new AnthropicMessagesStreamingResponse class that carries _hidden_params["additional_headers"], which the proxy's existing header-forwarding machinery picks up unchanged.

• Introduces AnthropicMessagesStreamingResponse — a thin async-iterator wrapper with a _hidden_params attribute built via the existing process_response_headers helper — and applies it to both the direct and agentic streaming return paths in async_anthropic_messages_handler.
• Adds a mock regression test that injects a Bedrock-shaped httpx.Response with x-amzn-* headers and asserts they appear as llm_provider-* prefixed entries in _hidden_params["additional_headers"] while SSE bytes pass through unchanged.
• Only the initial HTTP response's headers are captured; follow-up agentic requests do not surface their headers, which is an accepted limitation noted in the PR description.

Confidence Score: 4/5

The change is safe to merge: it adds a thin wrapper that the existing proxy header-forwarding machinery already knows how to read, and the non-streaming path is untouched.

The core fix is correct and well-targeted. The new AnthropicMessagesStreamingResponse class properly implements the async-iterator protocol and the _hidden_params dict is wired into the same plumbing that CustomStreamWrapper and the google_genai path already use. The test covers the primary (non-agentic) streaming path with real headers and byte assertions. The only gap is that the agentic branch — where AgenticAnthropicStreamingIterator is wrapped — has no dedicated test, so a regression there would be invisible in CI.

The agentic streaming path in llm_http_handler.py (lines 2115–2128) and the corresponding AnthropicMessagesStreamingResponse wrapping of AgenticAnthropicStreamingIterator deserve a second look, since that branch has no test coverage in this PR.

Important Files Changed

Filename	Overview
litellm/llms/anthropic/experimental_pass_through/messages/streaming_iterator.py	Adds AnthropicMessagesStreamingResponse wrapper class and anthropic_messages_stream_hidden_params helper to carry provider response headers; _RESPONSE_HEADERS_ADAPTER TypeAdapter validates the header dict at module level.
litellm/llms/custom_httpx/llm_http_handler.py	Wraps both the direct and agentic streaming paths in AnthropicMessagesStreamingResponse before returning; stream_hidden_params is built once from the first HTTP response's headers.
tests/test_litellm/llms/custom_httpx/test_llm_http_handler.py	New mock test validates that the returned stream carries llm_provider-prefixed headers and still yields SSE bytes; only the non-agentic (direct-passthrough) code path is exercised.

_{Reviews (2): Last reviewed commit: "fix(anthropic_messages): forward provide..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[mateo-berri]

@coderabbitai review

[mateo-berri]

bugbot run

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

• ✅ Fixed: Missing stream aclose forwarding
  • Added an aclose method on AnthropicMessagesStreamingResponse that forwards to the wrapped completion_stream's aclose so the proxy's streaming cleanup can tear down the upstream iterator.

_{You can send follow-ups to the cloud agent here.}

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ mateo-berri
❌ cursoragent
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[coderabbitai]

📝 Walkthrough

Walkthrough

Adds a new AnthropicMessagesStreamingResponse wrapper and anthropic_messages_stream_hidden_params helper that extract and store provider response headers as hidden params. Wires this wrapper into both agentic and non-agentic streaming branches of the Anthropic Messages HTTP handler, and adds a regression test.

Changes

Anthropic streaming header propagation

Layer / File(s)	Summary
Streaming response wrapper and header helper litellm/llms/anthropic/experimental_pass_through/messages/streaming_iterator.py	Adds AnthropicMessagesStreamHiddenParams TypedDict, anthropic_messages_stream_hidden_params() helper using process_response_headers and a TypeAdapter, and AnthropicMessagesStreamingResponse class wrapping an async byte stream while storing _hidden_params and implementing __aiter__/__anext__.
Handler wiring and regression test litellm/llms/custom_httpx/llm_http_handler.py, tests/test_litellm/llms/custom_httpx/test_llm_http_handler.py	Both the agentic-hook and non-agentic streaming branches now compute stream_hidden_params from response headers and return AnthropicMessagesStreamingResponse instead of the raw stream/iterator; a new async test verifies provider headers (e.g. x-amzn-requestid) surface in _hidden_params["additional_headers"] and stream chunks are preserved.

Estimated code review effort: 2 (Simple) | ~15 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Handler as BaseLLMHTTPHandler
  participant Helper as anthropic_messages_stream_hidden_params
  participant Wrapper as AnthropicMessagesStreamingResponse
  participant Stream as completion_stream/AgenticAnthropicStreamingIterator

  Handler->>Helper: extract headers from response.headers
  Helper-->>Handler: hidden_params (additional_headers)
  Handler->>Wrapper: new(completion_stream, hidden_params)
  Wrapper->>Stream: forward __anext__ calls
  Stream-->>Wrapper: bytes chunk
  Wrapper-->>Handler: async iterator with _hidden_params exposed

Loading

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly and concisely describes the main change: forwarding provider response headers on streaming Anthropic messages responses.
Description check	✅ Passed	The description matches the template well, with ticket info, checklist, proof of fix, type, and detailed changes; only the Relevant issues section is blank.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch litellm_anthropic_messages_stream_response_headers

---

_{Comment @coderabbitai help to get the list of available commands.}

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit af067b1. Configure here.}