fix(anthropic): preserve midturn system corrections

[eugene-yao-zocdoc]

TLDR

Claude Code can insert a correction as an in-sequence system message while a tool call is running. LiteLLM's Anthropic pass-through translators previously dropped that message, so the model continued with stale instructions.

This change:

• Preserves valid in-sequence system messages at their exact position when translating to OpenAI Chat Completions and Responses.
• Keeps those messages as system; it does not convert them to user.
• Treats in-sequence system content as untrusted client input for Anthropic /v1/messages guardrails.

User Flow

Before: the correction the user types mid-task never reaches the model, so the agent keeps executing the stale plan.

1. In a terminal, the user points Claude Code at the gateway (ANTHROPIC_BASE_URL=https://litellm-domain, ANTHROPIC_AUTH_TOKEN=<litellm key>) with their model set to an OpenAI-backed deployment, and launches claude.
2. They ask the agent to work through a multi-step task; each agent turn is a POST https://litellm-domain/v1/messages that returns tool calls the agent then runs.
3. While a tool call is still running, they type a correction ("stop doing X, do Y instead") and press Enter; Claude Code queues it and includes it in the next POST https://litellm-domain/v1/messages as an in-sequence system entry at the point in the conversation where it was typed.
4. The 200 reply continues the original task as if nothing was said: the gateway dropped the mid-turn system entry while translating the request for the OpenAI backend, so the model never saw the correction.
5. The user watches the agent keep following the stale instructions and has to abort the session by hand.

After: the same correction reaches the model at the same point in the conversation, and the agent changes course.

1. In a terminal, the user points Claude Code at the gateway (ANTHROPIC_BASE_URL=https://litellm-domain, ANTHROPIC_AUTH_TOKEN=<litellm key>) with their model set to an OpenAI-backed deployment, and launches claude.
2. They ask the agent to work through a multi-step task; each agent turn is a POST https://litellm-domain/v1/messages that returns tool calls the agent then runs.
3. While a tool call is still running, they type a correction ("stop doing X, do Y instead") and press Enter; Claude Code queues it and includes it in the next POST https://litellm-domain/v1/messages as an in-sequence system entry at the point in the conversation where it was typed.
4. The 200 reply acknowledges the correction and switches course: the mid-turn system entry is delivered to the OpenAI backend as a system message at the same position, kept separate from the trusted top-level system prompt.
5. The agent continues under the corrected instructions with no manual intervention, and any configured guardrails scan the correction text like any other client input.

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you are seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Behavior

OpenAI Chat Completions

A valid in-sequence Anthropic system entry becomes an OpenAI system message at the same sequence position. String and text-block content are supported. Text-block cache_control metadata follows the adapter's existing best-effort model-name handling.

OpenAI Responses

A valid in-sequence Anthropic system entry becomes a Responses message item with role: "system" and input_text content at the same position. Responses input text has no equivalent cache_control field, so that metadata is not forwarded.

Both translators omit empty, malformed, image-only, and otherwise unsupported system content.

Anthropic guardrails and compaction

For /v1/messages, the top-level system field remains the trusted prompt. In-sequence system entries are untrusted client input and are always included in guardrail text extraction and structured_messages, even when skip_system_message_in_guardrail skips the top-level prompt.

Guardrail write-back removes only the actual hoisted top-level prompt and preserves other in-sequence system entries around converted non-system runs. A system entry that lands between an assistant tool call and its tool results is emitted right after that exchange completes, so the call/result pair always converts together. Compaction token counting and summary construction also retain corrections in sequence.

Scope

This PR does not add provider detection, provider-specific rejection, fallback behavior, or changes to native Anthropic/Bedrock system-message handling. It defines preservation for the direct OpenAI Chat Completions and Responses translations and keeps corrections available through guardrail processing.

Type

Bug Fix

Tests

Final local verification on the affected files:

• 315 passed
• Ruff checks and format checks passed
• Strict Ruff budget gate passed
• Type-discipline gate passed
• basedpyright budget gate passed
• git diff --check passed

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world use-cases are not possible after this PR

Screenshots / Proof of Fix

E2E QA at PR head b9b200b (after) vs merge base 79d412e (before): the real Claude Code CLI v2.1.227 TUI, driven interactively under tmux, pointed at a LiteLLM proxy booted from each commit (own worktree, own venv, DB-less config, --detailed_debug as wire capture), model gpt-4o via the real OpenAI API, no mocks. Both legs ran the identical User Flow: ask the agent to run a slow foreground Bash command then write a haiku, and while the tool is still running type "Forget the haiku completely. Reply with exactly: CORRECTION RECEIVED - PINEAPPLE". Claude Code queued the correction and delivered it in the next POST /v1/messages as an in-sequence system entry right after the tool_result, exactly as this PR's TLDR claims

Before (79d412e): the correction is silently dropped and the user gets the stale haiku

.venv/bin/litellm --config qa_config.yaml --host 127.0.0.1 --port 33756 --detailed_debug

What the end user saw in the TUI:

❯ Use the Bash tool to run exactly: sleep 60 && echo done. Run it in the foreground, do NOT set run_in_background. Wait for it to finish, then write a haiku about the ocean.
⏺ Bash(sleep 60 && echo done)
  ⎿  done
❯ Forget the haiku completely. Reply with exactly: CORRECTION RECEIVED - PINEAPPLE
⏺ Ocean's waves whisper
  Under the moon's gentle glow
  Endless rhythms flow

On the wire, the incoming /v1/messages carried five messages ending in {'role': 'system', 'content': [{'type': 'text', 'text': 'The user sent a new message while you were working:\nForget the haiku completely. Reply with exactly: CORRECTION RECEIVED - PINEAPPLE ...'}]}, but the outgoing POST https://api.openai.com/v1/responses body contained zero occurrences of "PINEAPPLE" and zero system (or developer) roles: its input ended at the function_call_output and both in-sequence system entries (a SessionStart hook entry and the correction) were gone, with HTTP 200 and no warning

After (b9b200b): the same correction reaches OpenAI in place and the model obeys it

.venv/bin/litellm --config qa_config.yaml --host 127.0.0.1 --port 31375 --detailed_debug

What the end user saw in the TUI (this leg's Claude Code session refused foreground sleep with a tool error, so the slow foreground command was a 60s python sleep instead, which changes nothing about the claim under test):

❯ Use the Bash tool to run exactly: python3 -c "import time; time.sleep(60)" && echo done. Run it in the foreground, do NOT set run_in_background. Wait for it to finish, then write a haiku about the ocean.
  Ran 1 shell command
❯ Forget the haiku completely. Reply with exactly: CORRECTION RECEIVED - PINEAPPLE
⏺ CORRECTION RECEIVED - PINEAPPLE

Same incoming shape (five messages, correction as the in-sequence system entry right after the tool_result), and the outgoing OpenAI Responses request preserved all five positions: input[4] = {'type': 'message', 'role': 'system', 'content': [{'type': 'input_text', 'text': 'The user sent a new message while you were working:\nForget the haiku completely. Reply with exactly: CORRECTION RECEIVED - PINEAPPLE ...'}]} directly after the function_call_output, the SessionStart hook system entry (string content) preserved as input[1], and the top-level Claude Code system prompt sent exactly once as instructions. OpenAI streamed back exactly CORRECTION RECEIVED - PINEAPPLE

Caveats observed during QA (facts from the runs, not speculation from the diff):

• Out of the box this exact scenario 400s before the translation under test is even reached: Claude Code sends thinking by default, the Anthropic to Responses bridge maps it to reasoning.effort, and gpt-4o rejects that parameter; both legs needed client-side MAX_THINKING_TOKENS=0, and drop_params: true did not help on this path. Identical before and after, so this PR leaves it alone
• Both legs exercised the Anthropic to OpenAI Responses bridge, so this proof covers the Responses translation; the Chat Completions translation is covered by the PR's unit tests but was not the path this client took
• Before the PR the drop was broader than user corrections: every in-sequence system entry, including SessionStart hook output, was silently dropped with HTTP 200. The after leg showed both content shapes (string and text-block) preserved at position, so this PR fixes that too
• Opening the mid-turn window depends on client/model behavior this PR does not touch: gpt-4o sometimes picks run_in_background for the Bash tool (which ends the turn immediately), and one leg's Claude Code session hard-blocked foreground sleep while the concurrent other leg ran the identical command fine

[eugene-yao-zocdoc]

@greptileai

[greptile-apps]

Greptile Summary

Preserves in-sequence Anthropic system corrections throughout translation and guardrail processing.

• Converts supported corrections into position-preserving system messages for OpenAI Chat Completions and Responses.
• Scans and writes back untrusted in-sequence system content while keeping the trusted top-level prompt separate.
• Extends Anthropic message types and tests translation, masking, compaction, tool exchanges, and malformed content.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
litellm/llms/anthropic/chat/guardrail_translation/handler.py	Separates trusted top-level prompts from midturn system input during guardrail extraction and preserves supported corrections during structured-message rewrites.
litellm/llms/anthropic/experimental_pass_through/adapters/transformation.py	Adds Chat Completions translation for supported in-sequence system content while retaining its system role.
litellm/llms/anthropic/experimental_pass_through/responses_adapters/transformation.py	Maps supported in-sequence system content to Responses message items with input-text parts.
litellm/types/llms/anthropic.py	Introduces pass-through-specific typing for Anthropic in-sequence system entries.
tests/test_litellm/llms/anthropic/chat/guardrail_translation/test_anthropic_guardrail_handler.py	Adds extensive coverage for extraction, masking, structured rewrites, compaction, cache metadata, and tool-exchange handling.

_{Reviews (7): Last reviewed commit: "fix(anthropic): keep tool exchanges inta..." | Re-trigger Greptile}

[eugene-yao-zocdoc]

CI appears blocked by the stale daily base rather than this PR. The latest available branch is litellm_oss_daily_2026_07_20; neither the July 21 nor July 22 daily branch was created. The OpenAPI queued enum update and OSV dependency upgrades are already fixed on main. Could a maintainer cut the current daily OSS branch? I’ll rebase this PR once it’s available.

[eugene-yao-zocdoc]

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

[codecov]

Codecov Report

❌ Patch coverage is 96.55172% with 5 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...ms/anthropic/chat/guardrail_translation/handler.py	95.19%	5 Missing ⚠️

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing eugene-yao-zocdoc:litellm_fix_anthropic_midturn_interjections (b9b200b) with litellm_internal_staging (79d412e)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (6f36bee) during the generation of this report, so 79d412e was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 6b6f9f8. Configure here.}

[eugene-yao-zocdoc]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[mateo-berri]

bugbot run

[mateo-berri]

@greptileai

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit b9b200b. Configure here.}

[mateo-berri]

LGTM. Thanks for the contribution!