fix: passthrough endpoints duplicate logs

[mubashir1osmani]

Relevant issues

bedrock anthropic models use anthropic passthrough endpoints and stream is set to False (hardcoded) even when response is streamed in chunks.

success handler in logging does not identify this as a streaming response and completely misses the deduplication
guard we have in place

before:

after:

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Screenshots / Proof of Fix

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

[greptile-apps]

Greptile Summary

This PR fixes duplicate success-callback dispatches for streaming Anthropic passthrough endpoints. The root cause was that LiteLLMLoggingObj was always created with stream=False (hardcoded), so the dedup guard in _is_assembled_stream_success never activated for pass-through streaming requests.

• pass_through_endpoints.py: After _update_stream_param_based_on_request_body detects a streaming request, logging_obj.stream = True and model_call_details["stream"] = True are now set before entering the StreamingResponse path.
• anthropic_passthrough_logging_handler.py: complete_streaming_response is stored on model_call_details before the cost-calculation try block, so callbacks and the dedup guard see the assembled response even when pricing fails.
• Three regression tests are added; they operate entirely with local SSE frame construction and mocks (no real network calls).

Confidence Score: 4/5

The streaming-path fix is correctly scoped, but the non-streaming SSE fallback path does not receive the logging_obj.stream = True update, leaving a parallel dedup gap unaddressed.

The two production changes are internally consistent and safe. The remaining concern is that the non-streaming SSE fallback path (line 1144 in pass_through_endpoints.py) still initialises with logging_obj.stream = False and never updates it, so any caller reaching that path will not benefit from the dedup guard.

The non-streaming SSE fallback branch in pass_through_endpoints.py (around line 1144) does not set logging_obj.stream = True and warrants a follow-up.

Important Files Changed

Filename	Overview
litellm/proxy/pass_through_endpoints/pass_through_endpoints.py	Adds logging_obj.stream = True inside the if stream: block after _update_stream_param_based_on_request_body. Correctly patches the hardcoded stream=False in the Logging constructor for requests where streaming is detected via the request body. The SSE-fallback path at line 1144 still does not set logging_obj.stream = True, so the dedup guard remains disabled for that sub-path.
litellm/proxy/pass_through_endpoints/llm_provider_handlers/anthropic_passthrough_logging_handler.py	Sets complete_streaming_response on model_call_details before the cost-calculation try block. Ensures callbacks always receive the assembled response even when pricing fails, enabling the dedup guard to recognise the assembled state. Change is safe — litellm_model_response is guaranteed non-None at this call site.
tests/test_litellm/proxy/pass_through_endpoints/llm_provider_handlers/test_anthropic_passthrough_logging_handler.py	Adds three regression tests for the duplicate-callback bug using only mocks and local SSE frame construction (no real network calls). The dedup guard test validates conditions manually rather than calling dispatch_success_handlers twice.

_{Reviews (2): Last reviewed commit: "add" | Re-trigger Greptile}

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

[mubashir1osmani]

@greptile-apps review it

[codecov]

Codecov Report

❌ Patch coverage is 60.00000% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...y/pass_through_endpoints/pass_through_endpoints.py	50.00%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

[mubashir1osmani]

@greptile-apps added stream to logging obj along with a unit test. manual QA is done and this works as expected. tell me, are there any other endpoints we need to cover ?

[greptile-apps]

The stream=True fix in pass_through_endpoints.py applies globally to all providers — that part is already covered. However, the second half of the fix — setting complete_streaming_response on model_call_details before cost calculation — was only applied to the Anthropic handler.

The _is_assembled_stream_success guard requires both conditions:

1. self.stream is True ✅ (fixed globally)
2. self.model_call_details.get("complete_streaming_response") is not None ← still missing in these handlers

The following handlers build the assembled response but never store it in model_call_details["complete_streaming_response"]:

Handler	Method	Missing fix
vertex_passthrough_logging_handler.py	_handle_logging_vertex_collected_chunks → _create_vertex_response_logging_payload_for_generate_content	complete_streaming_response not set
gemini_passthrough_logging_handler.py	_handle_logging_gemini_collected_chunks → _create_gemini_response_logging_payload_for_generate_content	complete_streaming_response not set
openai_passthrough_logging_handler.py	_handle_logging_openai_collected_chunks	complete_streaming_response not set
base_passthrough_logging_handler.py	_handle_logging_llm_collected_chunks (used by Cohere)	complete_streaming_response not set

The simplest fix is the same one-liner you applied to the Anthropic handler — add it right after the assembled response is built in each of these methods:

litellm_logging_obj.model_call_details["complete_streaming_response"] = complete_streaming_response

The cleanest approach: add it in base_passthrough_logging_handler._handle_logging_llm_collected_chunks (covers Cohere automatically), then separately in the Vertex, Gemini, and OpenAI _handle_logging_*_collected_chunks since they don't go through the base class.

[mateo-berri]

This has been merged via internal copy. Thank you for the contribution!