fix(bedrock): keep mid-conversation system messages in place for Claude Invoke

[mateo-berri]

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

All runs are live end to end against real Bedrock (us.anthropic.claude-opus-4-8, us-west-2) with Claude Code as the client, costing real $. The proxy config maps claude-opus-4-8 to bedrock/us.anthropic.claude-opus-4-8 with aws_region_name: us-west-2, i.e. the plain bedrock/ prefix so /v1/messages traffic goes down the Invoke path

python litellm/proxy/proxy_cli.py --config repro_config.yaml --port 4000 --detailed_debug > proxy.log 2>&1

The client is a real Claude Code session that grows context from ~35k to ~250k input tokens across 13 API calls by reading three ~45k-token text files plus two small ones, all seconds apart (far under the 5m cache TTL):

ANTHROPIC_BASE_URL=http://127.0.0.1:4000 ANTHROPIC_AUTH_TOKEN=sk-repro-1234 \
claude -p "Read big1.txt fully, then big2.txt fully, then big3.txt fully, then small1.txt, then small2.txt. Use exactly one Read tool call at a time, waiting for each result before issuing the next. Do not summarize anything until all five files are read; after the fifth read, reply with exactly the word: done" \
  --allowedTools "Read" --model claude-opus-4-8

Per-call cache usage extracted from the message_start events in the debug log:

grep -o '"cache_read_input_tokens": *[0-9]*, "cache_creation": {"ephemeral_5m_input_tokens": *[0-9]*' proxy.log

Before, at 07aeaa1 (current litellm_internal_staging HEAD):

call	cache_read	cache_creation
1	0	37,312
2	33,436	35,328
3	68,764	283
4	69,047	25,975
5	95,022	12,062
6	33,436	105,078
7	138,514	25,986
8	164,500	12,053
9	33,436	174,727
10	208,163	25,961
11	234,124	12,056
12	246,180	2,719
13	248,899	2,736

Each bolded collapse coincides with Claude Code appending a new mid-conversation role: "system" message (a Read-truncation notice on call 6, a task-tool reminder on call 9), which the current code hoists into the top-level system field; the mutated system prefix invalidates the whole cached message history, so reads pin at the 33,436-token tools + system prefix and everything after re-writes at cache-write pricing

After, at b36ba99 (this branch), identical session:

call	cache_read	cache_creation
1	34,534	0
2	34,534	34,126
3	68,660	467
4	69,127	19,098
5	88,225	18,940
6	107,165	31,104
7	138,269	19,094
8	157,363	19,084
9	176,447	31,280
10	207,727	19,096
11	226,823	19,088
12	245,911	2,719
13	248,630	2,732

Reads grow monotonically with zero collapses and the outbound payloads carry the mid-conversation system entries in place, which Bedrock accepts with 200s (call 1 reads 34,534 because an identical warm-cache run had finished minutes earlier)

Version bisect with the same session against PyPI wheels: v1.89.1 (no hoist, passes role: "system" through in place) grows monotonically 33,435 -> 248,685 with no collapses; v1.91.0 (first release with the hoist) collapses from 202,378 back to 33,436 with a 212,403-token re-write on the turn a new system message arrives

Type

🐛 Bug Fix

Changes

#31364 fixed a real 400 (messages.0: use the top-level 'system' parameter for the initial system prompt) by hoisting every role: "system" entry from messages into the top-level system field on the Bedrock Invoke /v1/messages path. Bedrock's actual validation, verified live with direct invoke-model calls against us.anthropic.claude-opus-4-8, rejects a system entry only at messages.0 or when it neither immediately precedes an assistant message nor ends the array ("messages.N: role 'system' must precede an 'assistant' message or end the array"); mid-conversation system entries as Claude Code emits them via the mid-conversation-system-2026-04-07 beta (task reminders and tool-output truncation notices, inserted before an assistant turn or trailing) satisfy that rule and are accepted in place, which pre-hoist LiteLLM through v1.90.x relied on in production. Hoisting them is destructive for prompt caching: system precedes messages in the cache prefix, so every newly appended mid-conversation system message mutates system and invalidates the cache for the entire message history, re-writing 100k+ tokens at cache-write pricing on long agentic sessions

This PR hoists only the leading run of system entries (the case Invoke actually rejects) and forwards mid-conversation ones untouched. Billing-header stripping from top-level system previously only ran as a side effect of the hoist; it now applies regardless, so x-anthropic-billing-header blocks still never reach Bedrock

Two regression tests are added: one asserts a mid-conversation system message stays in place while the top-level system field stays byte-identical (this is the cache-collapse regression) and still strips billing-header blocks, the other asserts a mixed conversation hoists exactly the leading run and preserves the position of a later system entry. Both fail on the pre-fix code

Note that this is only a fix for Invoke because Converse (when empirically tested), rejects role: "system" in messages at any position. We have always hoisted in Converse since December 2024 (61b35c12bb)

[greptile-apps]

Greptile Summary

This PR narrows the Bedrock Invoke system-message normalization from hoisting every role: "system" entry in messages to hoisting only the leading run — the only case Bedrock actually rejects. Mid-conversation system entries (e.g. Claude Code's mid-conversation-system-2026-04-07 reminders) are now forwarded in place, preventing the prompt-cache invalidation that caused full re-writes on long agentic sessions.

• anthropic_claude3_transformation.py: replaces the "filter all system messages" list-comprehension with a next()-based leading-count probe and a conditional slice; billing-header stripping is promoted from a side-effect of the hoist to an unconditional step so it applies even when no leading system messages are present.
• test_anthropic_claude3_transformation.py: two new pure-unit regression tests cover the mid-conversation-in-place case (with billing-header filtering) and the mixed leading-run-plus-mid-conversation case; both are designed to fail on the pre-fix code.

Confidence Score: 5/5

The change is minimal, tightly scoped to one private method, backed by two targeted regression tests, and supported by live end-to-end billing evidence in the PR description.

The logic is straightforward: a next() scan replaces a full list comprehension, billing-header filtering is now unconditional (a strict improvement), and both new tests are mock-only unit tests that directly encode the failure modes.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/llms/bedrock/messages/invoke_transformations/anthropic_claude3_transformation.py	Fixes _normalize_system_role_messages_for_bedrock to hoist only the leading run of role: "system" messages instead of all of them; billing-header stripping now always applies regardless of whether anything was hoisted.
tests/test_litellm/llms/bedrock/messages/invoke_transformations/test_anthropic_claude3_transformation.py	Adds two new regression tests: one verifies mid-conversation system messages stay in place while billing-header blocks are still stripped, the other verifies only the leading run is hoisted in a mixed conversation. No network calls.

_{Reviews (1): Last reviewed commit: "fix(bedrock): keep mid-conversation syst..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 30 untouched benchmarks

---

_{Comparing litellm_bedrock_invoke_keep_mid_conversation_system (b36ba99) with litellm_internal_staging (142d5aa)}