Skip to content

fix(anthropic): keep input_transformations in the logged response - #41383

Open
clonylu wants to merge 1 commit into
BerriAI:mainfrom
clonylu:fix/anthropic-log-input-transformations
Open

clonylu wants to merge 1 commit into
BerriAI:mainfrom
clonylu:fix/anthropic-log-input-transformations

Conversation

@clonylu

@clonylu clonylu commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • Anthropic's input_transformations drop report never reaches spend logs or s3_v2
  • Lost on /v1/messages and /v1/chat/completions, streaming and not

How it solves it:

  • Copy the field into message.provider_specific_fields for parsed responses
  • Carry it from message_start into the assembled stream
  • Keep [], it proves the beta reached the provider

User Flow

Before:

  1. An operator enables thinking-binding-controls-2026-08-01 with prefix_mismatch_behavior: drop_block on a Claude Fable 5.1 deployment
  2. A client replays an assistant turn after editing an earlier message and gets HTTP 200 with input_transformations: [{"type": "thinking_dropped", ...}]
  3. The request's s3_v2 record has no input_transformations at all, so the edit and the lost reasoning are invisible in the logs

After:

  1. Same setup and request
  2. Same client response
  3. The logged message.provider_specific_fields.input_transformations is the provider's array as returned: the thinking_dropped entry for the edited replay, [] for a clean turn, and absent when the beta header is not sent. This also holds when streaming, and on /v1/chat/completions against an anthropic/ deployment the client now sees the field too

Relevant issues

Fixes #41382

Related: #41203 registers the beta header, and #41362 keeps the signed thinking blocks the binding check needs

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Tests: test_anthropic_chat_transformation.py, test_anthropic_chat_handler.py and test_anthropic_passthrough_logging_handler.py pass (491 passed). The 6 new cases asserting the field is present fail on unpatched main, for a thinking_dropped entry and for []. scripts/check_type_discipline.py counts are unchanged in the three library files

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Shared setup: a litellm --config proxy run from source with two deployments, bedrock/us.anthropic.claude-fable-5-1 (via aws_bedrock_runtime_endpoint) and anthropic/claude-fable-5-1 (via api_base), both pointed at a stand-in upstream that returns the response shape Claude Fable 5.1 returns with the beta. That is input_transformations: [] on a first turn, [{"type": "thinking_dropped", "path": "messages.1.content.0", "reason": "prefix_binding_mismatch"}] on a replayed turn, and in streams the field on message_start.message. A CustomLogger records standard_logging_object.response, which is what s3_v2 writes. Every request sends anthropic-beta: thinking-binding-controls-2026-08-01

Before (6dbd65b)

/v1/messages

  1. anthropic/ replayed turn, non-stream: the client gets the thinking_dropped entry, and the logged provider_specific_fields keys are citations, thinking_blocks with no input_transformations anywhere in the record
  2. anthropic/ replayed turn, stream: the client's message_start has the entry, and the logged keys are thinking_blocks only
  3. anthropic/ first turn, non-stream and stream: the client gets [], and the record has no input_transformations
  4. bedrock/ invoke, replayed and first turn, non-stream: same as 1 and 3

/v1/chat/completions (anthropic/)

  1. Non-stream: neither the client message nor the logged record has the field
  2. Stream: no chunk has it, and the logged keys are thinking_blocks only

After (588f152)

/v1/messages

  1. anthropic/ replayed turn, non-stream: the logged keys are citations, input_transformations, thinking_blocks, and input_transformations is the same thinking_dropped entry the client got
  2. anthropic/ replayed turn, stream: the logged keys are input_transformations, thinking_blocks, with the same entry
  3. anthropic/ first turn, non-stream and stream: the record logs input_transformations: []
  4. bedrock/ invoke, replayed and first turn, non-stream: same as 1 and 3, and each record has the key exactly once

/v1/chat/completions (anthropic/)

  1. Non-stream: the client message.provider_specific_fields.input_transformations and the logged record both have the entry
  2. Stream: the first chunk's delta.provider_specific_fields has it once, and stream_chunk_builder puts it on the logged message

The stand-in replays shapes recorded from Claude Fable 5.1 on Bedrock and Vertex AI through a proxy on v1.101.0-rc.1 with the beta header. There the edited replay's s3_v2 record had no input_transformations, the same as Before row 1. The fix copies the provider's value without reshaping it

Type

🐛 Bug Fix

Caveats (if any)

Low

  • Streaming chat now emits a leading message_start chunk carrying the field
    • Same as existing web_search_results and thinking_blocks chunks, and only when the beta header is sent
  • The Rust-bridge logging fallback still drops the field
    • It already drops context_management and container the same way, so it is left as is

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

@clonylu
clonylu requested a review from a team September 16, 2026 07:09
@greptile-apps

greptile-apps Bot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Preserves Anthropic input_transformations in provider-specific response metadata for nonstream responses and message_start streaming events

  • Presence checks retain both dropped-thinking reports and empty arrays while leaving responses without the field unchanged
  • New tests cover parsed responses, streaming chunks, and both passthrough stream assembly implementations
  • The added dictionary assignments need adjustment to satisfy the repository’s immutable-code requirement

Confidence Score: 4/5

The response-forwarding behavior appears correct, but the explicit immutable-code requirement must be satisfied before merging

No functional or security defect was established in the scoped HTTP paths; the accepted finding concerns new dictionary mutations prohibited by the repository guide

Files Needing Attention: litellm/llms/anthropic/chat/handler.py; litellm/llms/anthropic/chat/transformation.py

Important Files Changed
Filename Overview
litellm/llms/anthropic/chat/handler.py Preserves input_transformations from message_start; the new dictionary mutation violates the repository guide
litellm/llms/anthropic/chat/transformation.py Copies present input_transformations verbatim, including empty arrays; the same mutation requirement applies
litellm/types/llms/anthropic.py Adds an optional read-only input_transformations field to MessageChunk
tests/test_litellm/llms/anthropic/chat/test_anthropic_chat_handler.py Covers populated, empty, and absent metadata on message_start
tests/test_litellm/llms/anthropic/chat/test_anthropic_chat_transformation.py Covers populated, empty, and absent metadata in parsed responses
tests/test_litellm/proxy/pass_through_endpoints/llm_provider_handlers/test_anthropic_passthrough_logging_handler.py Checks metadata preservation and unchanged text through both stream assembly implementations

Reviews (1): Last reviewed commit: "fix(anthropic): keep input_transformatio..." | Re-trigger Greptile

@codspeed

codspeed Bot commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing clonylu:fix/anthropic-log-input-transformations (588f152) with main (09ebb28)

Open in CodSpeed

Comment on lines +1073 to +1075
provider_specific_fields["input_transformations"] = message_start_block["message"][
"input_transformations"
]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 New dictionary mutation

Assignments here and in transformation.py violate CLAUDE.md’s no-mutation directive. Both must use immutable construction before merging

Context Used: CLAUDE.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both lines follow the existing construction of these two dicts: _build_provider_specific_fields already sets context_management, web_search_results, tool_results, code_interpreter_results, container and compaction_blocks on provider_specific_fields the same way in the lines around this one, and the chunk_parser branch does the same for thinking_blocks / compaction_delta. check_type_discipline.py per-rule counts are unchanged by this PR. Rewriting one key immutably inside builders that mutate the same dict a dozen times would leave the function inconsistent, and refactoring both builders is out of scope for a logging fix; happy to do that as a follow-up if a maintainer wants it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That context is fair. I agree this is not an actionable finding: both assignments follow the established mutation pattern in the surrounding builders, and making only these additions immutable would be inconsistent without refactoring the existing construction. The unchanged per-rule counts also support that this PR does not introduce a new mutation category. I’ll withdraw this comment; a broader immutable-builder refactor can be handled separately if desired.

Tip: You can customize Greptile's behavior for this repo with .greptile/rules.md and .greptile/config.json.

@codecov

codecov Bot commented Sep 16, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Anthropic's thinking-binding beta (thinking-binding-controls-2026-08-01)
adds a top-level `input_transformations` field to a Messages response:
`[]` when every replayed thinking block bound to its prefix, and one entry
per block the provider removed under
`thinking.block_binding.prefix_mismatch_behavior: drop_block`, e.g.
`{"type": "thinking_dropped", "path": "messages.1.content.0",
"reason": "prefix_binding_mismatch"}`. It is the only signal that a client
edited the prefix and lost the model's reasoning, so operators need it in
the gateway's logs, not just in the client response.

On /v1/messages LiteLLM forwarded it to the client but dropped it from the
ModelResponse it logs; on /v1/chat/completions neither the client nor the
log saw it. `_build_provider_specific_fields` never copied it into
`provider_specific_fields`, and `chunk_parser` read only `usage` from
`message_start`, which is where the field lives in a stream. The standard
logging payload (spend logs, s3_v2, custom loggers) therefore had no trace
of it on either route, streaming or not.

- non-streaming: `_build_provider_specific_fields` forwards
  `input_transformations` next to `context_management`
- streaming: `chunk_parser` emits it on the `message_start` chunk's delta,
  so `stream_chunk_builder` -- and with it both passthrough builders and
  the CustomStreamWrapper path -- carries it into
  `message.provider_specific_fields`
- `[]` is kept: it proves the header reached the provider and nothing was
  dropped
- `MessageChunk` declares the field so the TypedDict access is typed

Streaming caveat: on /v1/chat/completions the `message_start` chunk now
has a non-empty delta.provider_specific_fields when the beta is in use, so
it reaches the client as a leading chunk instead of being swallowed as
empty (the same behaviour web_search_results / thinking chunks have).
@clonylu
clonylu force-pushed the fix/anthropic-log-input-transformations branch from aa5eb11 to 588f152 Compare September 24, 2026 07:52

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant