Skip to content

fix(logging): pass through Response*Event when /v1/messages bridges OpenAI Responses API - #29413

Closed
silencedoctor wants to merge 1 commit into
BerriAI:mainfrom
silencedoctor:fix/anthropic-messages-stream-response-events
Closed

fix(logging): pass through Response*Event when /v1/messages bridges OpenAI Responses API#29413
silencedoctor wants to merge 1 commit into
BerriAI:mainfrom
silencedoctor:fix/anthropic-messages-stream-response-events

Conversation

@silencedoctor

Copy link
Copy Markdown
Contributor

Relevant issues

Fixes #28595
Fixes #28943

Changes

When /v1/messages is routed to a backend that returns OpenAI Responses API stream events instead of native Anthropic shape (e.g. an OpenAI-Responses-compatible Ollama backend), the success handler crashes inside AnthropicResponse.model_validate(result). The exception is caught by _success_handler_helper_fn as [Non-Blocking], so the client gets a normal streaming response — but the spend_logs row is silently dropped and the UI shows no entry.

AnthropicResponsesStreamWrapper.async_anthropic_sse_wrapper already translates the events back to anthropic-shape SSE for the client. The logging path does not need to re-translate; passing the raw event through preserves usage fields for downstream cost tracking.

This PR adds an early return in _handle_anthropic_messages_response_logging for streaming requests when result is a ResponseCompletedEvent / ResponseIncompleteEvent / ResponseFailedEvent, and updates the return type annotation from ModelResponse to Any to match the existing branches that return non-ModelResponse objects.

Non-streaming requests are unaffected — a Response*Event arriving on a non-streaming path is still unexpected and continues into the existing AnthropicResponse.model_validate branch.

Testing

  • Added a parametrized regression test covering all three event types (ResponseCompletedEvent, ResponseIncompleteEvent, ResponseFailedEvent): with stream=True and call_type=\"anthropic_messages\", the handler returns the event untouched and never calls AnthropicResponse.model_validate
  • Added a guard test that the non-streaming path still flows into AnthropicResponse.model_validate for Response*Event inputs (preventing accidental over-swallowing)
  • Verified the parametrized cases fail without this fix (drop the patch from litellm_logging.py, re-run, all three fail — proving the tests aren't no-ops)
  • Ran `uv run pytest tests/test_litellm/litellm_core_utils/test_litellm_logging.py` — 86 passed

Type

🐛 Bug Fix

…penAI Responses API

When /v1/messages is routed to a backend that returns OpenAI Responses
API stream events instead of native Anthropic shape, the success handler
crashes inside AnthropicResponse.model_validate(result), which is then
caught as "[Non-Blocking]" and silently drops the spend_logs row.

The client-facing stream is already translated to anthropic-shape SSE by
AnthropicResponsesStreamWrapper, so the logging path can safely return
the raw event untouched. Add an early return in
_handle_anthropic_messages_response_logging for streaming requests when
result is a ResponseCompletedEvent / ResponseIncompleteEvent /
ResponseFailedEvent.

Fixes BerriAI#28595
Fixes BerriAI#28943
@codspeed-hq

codspeed-hq Bot commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 16 untouched benchmarks


Comparing silencedoctor:fix/anthropic-messages-stream-response-events (f9101bf) with main (5be0797)

Open in CodSpeed

@greptile-apps

greptile-apps Bot commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes a silent logging failure in _handle_anthropic_messages_response_logging when /v1/messages is bridged to an OpenAI Responses API backend. Previously, a streaming ResponseCompletedEvent/ResponseIncompleteEvent/ResponseFailedEvent would crash in AnthropicResponse.model_validate, the exception would be swallowed as [Non-Blocking], and all downstream logging callbacks (including spend tracking) would be skipped entirely.

  • Adds an early-return guard in _handle_anthropic_messages_response_logging so streaming Response*Event objects bypass model_validate and are passed through untouched; the existing _get_assembled_streaming_response path already handles these types for usage extraction.
  • Updates the return type from ModelResponse to Any to accurately reflect all existing branches.
  • Adds a parametrized regression test for all three event types and a guard test that confirms non-streaming requests still enter the model_validate branch.

Confidence Score: 5/5

Safe to merge — the change is a minimal guard that prevents a pre-existing crash in a single method, and the non-streaming path is unchanged.

The fix is a two-line guard that mirrors the adjacent pattern already present in the same function. The imports for the three event types have existed since earlier commits. The streaming cost-tracking path already handles these event types correctly; this change simply prevents the crash that was aborting the logging chain before that path could be reached. Tests cover all three event types and verify the non-streaming guard is not over-applied.

No files require special attention.

Important Files Changed

Filename Overview
litellm/litellm_core_utils/litellm_logging.py Adds an early return in _handle_anthropic_messages_response_logging for streaming Response*Event objects, preventing a crash in AnthropicResponse.model_validate when the /v1/messages route bridges through the OpenAI Responses API. Updates the return type annotation from ModelResponse to Any to match all existing return branches.
tests/test_litellm/litellm_core_utils/test_litellm_logging.py Adds a parametrized regression test covering all three Response*Event types in the streaming path, and a guard test that confirms non-streaming requests still route through AnthropicResponse.model_validate. All tests use mocks and make no real network calls.

Reviews (1): Last reviewed commit: "fix(logging): pass through Response*Even..." | Re-trigger Greptile

@codecov

codecov Bot commented Jun 1, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 75.00000% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
litellm/litellm_core_utils/litellm_logging.py 75.00% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

@Sameerlite

Copy link
Copy Markdown
Contributor

@silencedoctor — could you add a screenshot or short video showing that this change works as expected? It really helps reviewers verify the fix quickly. Thanks!

@silencedoctor

Copy link
Copy Markdown
Contributor Author

@Sameerlite
I used the same local repro setup for both runs: POST /v1/messages with
stream: true, routed through the OpenAI Responses API bridge using a local
Responses-compatible backend.

Before fix: parent commit 06f6cfc5. The client stream still completes
and returns message_stop, but LiteLLM success logging fails. The proxy logs
show ResponseCompletedEvent being passed into
AnthropicResponse.model_validate, causing the non-blocking success callback
error, and the custom success callback does not fire.

After fix: PR head f9101bf. The same request still completes on the
client side, and the LiteLLM success callback now fires successfully with
call_type=anthropic_messages and response_class=ResponsesAPIResponse.

So the screenshots show the actual issue clearly: this PR does not change the
client stream behavior; it fixes the broken success logging chain for /v1/ messages streaming through the Responses API bridge.
pr29413_before_repro_screenshot

pr29413_after_repro_screenshot

@piotrminkina

Copy link
Copy Markdown
Contributor

Compare with my PR #28985

@silencedoctor

Copy link
Copy Markdown
Contributor Author

fix #28985

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants