Repository navigation
Conversation
Greptile SummaryPreserves Anthropic input_transformations in provider-specific response metadata for nonstream responses and message_start streaming events
Confidence Score: 4/5The response-forwarding behavior appears correct, but the explicit immutable-code requirement must be satisfied before merging No functional or security defect was established in the scoped HTTP paths; the accepted finding concerns new dictionary mutations prohibited by the repository guide Files Needing Attention: litellm/llms/anthropic/chat/handler.py; litellm/llms/anthropic/chat/transformation.py Important Files Changed
Reviews (1): Last reviewed commit: "fix(anthropic): keep input_transformatio..." | Re-trigger Greptile |
| provider_specific_fields["input_transformations"] = message_start_block["message"][ | ||
| "input_transformations" | ||
| ] |
There was a problem hiding this comment.
Assignments here and in transformation.py violate CLAUDE.md’s no-mutation directive. Both must use immutable construction before merging
Context Used: CLAUDE.md (source)
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
There was a problem hiding this comment.
Both lines follow the existing construction of these two dicts: _build_provider_specific_fields already sets context_management, web_search_results, tool_results, code_interpreter_results, container and compaction_blocks on provider_specific_fields the same way in the lines around this one, and the chunk_parser branch does the same for thinking_blocks / compaction_delta. check_type_discipline.py per-rule counts are unchanged by this PR. Rewriting one key immutably inside builders that mutate the same dict a dozen times would leave the function inconsistent, and refactoring both builders is out of scope for a logging fix; happy to do that as a follow-up if a maintainer wants it.
There was a problem hiding this comment.
That context is fair. I agree this is not an actionable finding: both assignments follow the established mutation pattern in the surrounding builders, and making only these additions immutable would be inconsistent without refactoring the existing construction. The unchanged per-rule counts also support that this PR does not introduce a new mutation category. I’ll withdraw this comment; a broader immutable-builder refactor can be handled separately if desired.
Tip: You can customize Greptile's behavior for this repo with .greptile/rules.md and .greptile/config.json.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Anthropic's thinking-binding beta (thinking-binding-controls-2026-08-01)
adds a top-level `input_transformations` field to a Messages response:
`[]` when every replayed thinking block bound to its prefix, and one entry
per block the provider removed under
`thinking.block_binding.prefix_mismatch_behavior: drop_block`, e.g.
`{"type": "thinking_dropped", "path": "messages.1.content.0",
"reason": "prefix_binding_mismatch"}`. It is the only signal that a client
edited the prefix and lost the model's reasoning, so operators need it in
the gateway's logs, not just in the client response.
On /v1/messages LiteLLM forwarded it to the client but dropped it from the
ModelResponse it logs; on /v1/chat/completions neither the client nor the
log saw it. `_build_provider_specific_fields` never copied it into
`provider_specific_fields`, and `chunk_parser` read only `usage` from
`message_start`, which is where the field lives in a stream. The standard
logging payload (spend logs, s3_v2, custom loggers) therefore had no trace
of it on either route, streaming or not.
- non-streaming: `_build_provider_specific_fields` forwards
`input_transformations` next to `context_management`
- streaming: `chunk_parser` emits it on the `message_start` chunk's delta,
so `stream_chunk_builder` -- and with it both passthrough builders and
the CustomStreamWrapper path -- carries it into
`message.provider_specific_fields`
- `[]` is kept: it proves the header reached the provider and nothing was
dropped
- `MessageChunk` declares the field so the TypedDict access is typed
Streaming caveat: on /v1/chat/completions the `message_start` chunk now
has a non-empty delta.provider_specific_fields when the beta is in use, so
it reaches the client as a leading chunk instead of being swallowed as
empty (the same behaviour web_search_results / thinking chunks have).
aa5eb11 to
588f152
Compare
TLDR
Problem this solves:
input_transformationsdrop report never reaches spend logs ors3_v2/v1/messagesand/v1/chat/completions, streaming and notHow it solves it:
message.provider_specific_fieldsfor parsed responsesmessage_startinto the assembled stream[], it proves the beta reached the providerUser Flow
Before:
thinking-binding-controls-2026-08-01withprefix_mismatch_behavior: drop_blockon a Claude Fable 5.1 deploymentinput_transformations: [{"type": "thinking_dropped", ...}]s3_v2record has noinput_transformationsat all, so the edit and the lost reasoning are invisible in the logsAfter:
message.provider_specific_fields.input_transformationsis the provider's array as returned: thethinking_droppedentry for the edited replay,[]for a clean turn, and absent when the beta header is not sent. This also holds when streaming, and on/v1/chat/completionsagainst ananthropic/deployment the client now sees the field tooRelevant issues
Fixes #41382
Related: #41203 registers the beta header, and #41362 keeps the signed thinking blocks the binding check needs
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more@greptileaito re-request a review after pushing changes)Tests:
test_anthropic_chat_transformation.py,test_anthropic_chat_handler.pyandtest_anthropic_passthrough_logging_handler.pypass (491 passed). The 6 new cases asserting the field is present fail on unpatchedmain, for athinking_droppedentry and for[].scripts/check_type_discipline.pycounts are unchanged in the three library filesDelays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Shared setup: a
litellm --configproxy run from source with two deployments,bedrock/us.anthropic.claude-fable-5-1(viaaws_bedrock_runtime_endpoint) andanthropic/claude-fable-5-1(viaapi_base), both pointed at a stand-in upstream that returns the response shape Claude Fable 5.1 returns with the beta. That isinput_transformations: []on a first turn,[{"type": "thinking_dropped", "path": "messages.1.content.0", "reason": "prefix_binding_mismatch"}]on a replayed turn, and in streams the field onmessage_start.message. ACustomLoggerrecordsstandard_logging_object.response, which is whats3_v2writes. Every request sendsanthropic-beta: thinking-binding-controls-2026-08-01Before (6dbd65b)
/v1/messages
anthropic/replayed turn, non-stream: the client gets thethinking_droppedentry, and the loggedprovider_specific_fieldskeys arecitations,thinking_blockswith noinput_transformationsanywhere in the recordanthropic/replayed turn, stream: the client'smessage_starthas the entry, and the logged keys arethinking_blocksonlyanthropic/first turn, non-stream and stream: the client gets[], and the record has noinput_transformationsbedrock/invoke, replayed and first turn, non-stream: same as 1 and 3/v1/chat/completions (
anthropic/)thinking_blocksonlyAfter (588f152)
/v1/messages
anthropic/replayed turn, non-stream: the logged keys arecitations,input_transformations,thinking_blocks, andinput_transformationsis the samethinking_droppedentry the client gotanthropic/replayed turn, stream: the logged keys areinput_transformations,thinking_blocks, with the same entryanthropic/first turn, non-stream and stream: the record logsinput_transformations: []bedrock/invoke, replayed and first turn, non-stream: same as 1 and 3, and each record has the key exactly once/v1/chat/completions (
anthropic/)message.provider_specific_fields.input_transformationsand the logged record both have the entrydelta.provider_specific_fieldshas it once, andstream_chunk_builderputs it on the logged messageThe stand-in replays shapes recorded from Claude Fable 5.1 on Bedrock and Vertex AI through a proxy on v1.101.0-rc.1 with the beta header. There the edited replay's
s3_v2record had noinput_transformations, the same as Before row 1. The fix copies the provider's value without reshaping itType
🐛 Bug Fix
Caveats (if any)
Low
message_startchunk carrying the fieldweb_search_resultsandthinking_blockschunks, and only when the beta header is sentcontext_managementandcontainerthe same way, so it is left as isFinal Attestation