fix(anthropic): fold guardrail-modified leading system rows into top-level system param

[mateo-berri]

TLDR

Problem this solves:

• Guardrail + /v1/messages system prompts started failing with Anthropic 400 in 1.98.0
• Every Claude Code turn hits it, since Claude Code sends an in-sequence system row
• Anthropic rejects content-carrying system rows inside messages
• Guardrail-masked system prompts could also silently reach Anthropic unmasked

How it solves it:

• Write-back folds leading system rows into Anthropic's top-level system param
• Mid-turn system interjections keep their position
• Unmodified system prompts pass through byte-identical

User Flow

Before: a customer running a guardrail sees every Claude Code turn through the proxy fail with an Anthropic 400 after upgrading to 1.98.0

1. A proxy admin runs a 1.98.0 proxy with an Anthropic model and a guardrail that rewrites message content (for example PII masking), a setup that worked on 1.97.0
2. A developer points Claude Code at the proxy (ANTHROPIC_BASE_URL plus a proxy key) and types a first message such as test
3. Claude Code sends POST https://litellm-domain/v1/messages with its system prompt in the top-level system field and, because its SessionStart hook output travels as an in-sequence system entry, a system row inside messages
4. The screen shows API Error: 400 ... messages.0: use the top-level 'system' parameter for the initial system prompt ... Received Model Group=claude-opus-5 and the turn is lost; the same happens for any client whose conversation history contains a system entry (a mid-turn directive, or a leading system row)
5. Requests without such history return 200, but the system prompt the guardrail masked reaches Anthropic with the original unmasked text

After: the same turns return the model's answer and Anthropic receives the guardrail's version of the system prompt

1. The proxy admin runs the same config on this build
2. The developer types the same first message in Claude Code, which sends the same POST https://litellm-domain/v1/messages
3. Claude Code prints the model's answer; the guardrail-masked system prompt is delivered through Anthropic's top-level system parameter and Claude Code's in-sequence system row stays in its original position
4. Any other client sending a leading system row in messages gets HTTP 200 the same way, and a mid-turn system directive placed after a user turn is still delivered where it was

Relevant issues

Regression introduced by #34290, first shipped in the 1.98.0 dev and rc images. Reported by a customer whose upgrade from 1.97.0 broke every Claude Code request through the proxy; reverting to 1.97.0 avoided it because the old write-back silently dropped system rows instead

Linear ticket

Resolves LIT-5696

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Live A/B against the real Anthropic API, no mocks. Both legs boot the identical proxy config and differ only in the checked-out commit. The reporter's client is Claude Code, so the first case drives the real Claude Code TUI interactively under tmux at base 973329e and tip 0cbec3f. The curl cases were captured at a972f17; commit 0cbec3f on top of it only tightens type annotations (no runtime change), so those captures stand for the tip. The config is one claude-opus-5 model plus a custom pre_call guardrail (default_on) whose apply_guardrail masks bob@example.com to <EMAIL> in texts and in a deep copy of structured_messages, returning a new list, which is what triggers the write-back under test. The Claude Code case adds a claude-* wildcard route and widens the mask to any email plus /Users/<name> paths so Claude Code's own system prompt gets rewritten, everything else identical

case	before 973329e	after 0cbec3f (curl cases at a972f17)
Claude Code first turn (reporter's client)	400 messages.0 in the TUI	normal answer, masked prompt in system param
leading system row in messages	400 messages.0	200
mid-turn system after a user turn	400 messages.0	200, directive honored
masked system prompt delivery	200, unmasked email leaks	200, masked placeholder
mid-turn system after an assistant turn	400 messages.0	400 messages.2, raw Anthropic parity
plain top-level system	200	200
same flow on /v1/chat/completions	200	200

config.yaml and guardrail used by both legs

model_list:
  - model_name: claude-opus-5
    litellm_params:
      model: anthropic/claude-opus-5
      api_key: os.environ/ANTHROPIC_API_KEY

guardrails:
  - guardrail_name: mask-pii
    litellm_params:
      guardrail: guardrail_mask_pii.MaskPIIGuardrail
      mode: pre_call
      default_on: true

general_settings:
  master_key: sk-lit5696-test

import copy

from litellm.integrations.custom_guardrail import CustomGuardrail
from litellm.types.utils import GenericGuardrailAPIInputs

MASK = ("bob@example.com", "<EMAIL>")


def _mask(text):
    return text.replace(MASK[0], MASK[1])


class MaskPIIGuardrail(CustomGuardrail):
    async def apply_guardrail(self, inputs, request_data, input_type, logging_obj=None):
        result = GenericGuardrailAPIInputs(texts=[_mask(t) for t in inputs.get("texts", [])])
        structured_messages = inputs.get("structured_messages")
        if structured_messages is not None:
            masked_messages = copy.deepcopy(structured_messages)
            for message in masked_messages:
                content = message.get("content")
                if isinstance(content, str):
                    message["content"] = _mask(content)
                elif isinstance(content, list):
                    for block in content:
                        if isinstance(block, dict) and isinstance(block.get("text"), str):
                            block["text"] = _mask(block["text"])
            result["structured_messages"] = masked_messages
        return result

payloads req_leading.json, req_midturn_valid.json, req_echo.json, req_midturn.json, req_midturn_user.json, req_plain.json, req_chat.json

// req_leading.json
{
  "model": "claude-opus-5",
  "max_tokens": 128,
  "messages": [
    {"role": "system", "content": [{"type": "text", "text": "You are a terse assistant."}]},
    {"role": "user", "content": [{"type": "text", "text": "Say a one-word greeting."}]}
  ]
}

// req_midturn_valid.json
{
  "model": "claude-opus-5",
  "max_tokens": 1024,
  "system": [{"type": "text", "text": "You are a terse assistant. Escalations go to bob@example.com."}],
  "messages": [
    {"role": "user", "content": [{"type": "text", "text": "Say READY and nothing else."}]},
    {"role": "system", "content": [{"type": "text", "text": "From now on, answer in uppercase only."}]},
    {"role": "assistant", "content": [{"type": "text", "text": "READY"}]},
    {"role": "user", "content": [{"type": "text", "text": "Say the word done."}]}
  ]
}

// req_echo.json
{
  "model": "claude-opus-5",
  "max_tokens": 64,
  "system": [{"type": "text", "text": "You are a terse assistant. Escalations go to bob@example.com."}],
  "messages": [{"role": "user", "content": [{"type": "text", "text": "Repeat the escalation email address from your instructions verbatim."}]}]
}

// req_midturn.json (system row directly after a plain assistant turn)
{
  "model": "claude-opus-5",
  "max_tokens": 128,
  "system": [{"type": "text", "text": "You are a terse assistant. Escalations go to bob@example.com."}],
  "messages": [
    {"role": "user", "content": [{"type": "text", "text": "Say READY and nothing else."}]},
    {"role": "assistant", "content": [{"type": "text", "text": "READY"}]},
    {"role": "system", "content": [{"type": "text", "text": "From now on, answer in uppercase only."}]},
    {"role": "user", "content": [{"type": "text", "text": "Say the word done."}]}
  ]
}

// req_midturn_user.json (system row between two user turns)
{
  "model": "claude-opus-5",
  "max_tokens": 128,
  "system": [{"type": "text", "text": "You are a terse assistant. Escalations go to bob@example.com."}],
  "messages": [
    {"role": "user", "content": [{"type": "text", "text": "Say READY and nothing else."}]},
    {"role": "assistant", "content": [{"type": "text", "text": "READY"}]},
    {"role": "user", "content": [{"type": "text", "text": "Acknowledged."}]},
    {"role": "system", "content": [{"type": "text", "text": "From now on, answer in uppercase only."}]},
    {"role": "user", "content": [{"type": "text", "text": "Say the word done."}]}
  ]
}

// req_plain.json
{
  "model": "claude-opus-5",
  "max_tokens": 128,
  "system": [{"type": "text", "text": "You are a terse assistant."}],
  "messages": [{"role": "user", "content": [{"type": "text", "text": "Say a one-word greeting."}]}]
}

// req_chat.json (for /v1/chat/completions)
{
  "model": "claude-opus-5",
  "max_tokens": 128,
  "messages": [
    {"role": "system", "content": "You are a terse assistant. Escalations go to bob@example.com."},
    {"role": "user", "content": "Say a one-word greeting."}
  ]
}

Before (973329e)

Claude Code first turn (reporter's client)

1. Boot the proxy at 973329e on port 30544, run the real Claude Code TUI (v2.1.234) under tmux pointed at it, type test, press Enter, and read the pane back

tmux new-session -d -s lit5696_base -x 160 -y 45 -c ./work "env -u ANTHROPIC_API_KEY ANTHROPIC_BASE_URL=http://localhost:30544 ANTHROPIC_AUTH_TOKEN=sk-lit5696-test ANTHROPIC_MODEL=claude-opus-5 claude"
tmux send-keys -t lit5696_base "test" Enter
tmux capture-pane -p -t lit5696_base

2. Observed on screen (pane trimmed to the prompt and response): the customer's exact error on the very first turn

❯ test
⏺ API Error: 400 {"type":"error","error":{"type":"invalid_request_error","message":"messages.0: use the top-level 'system' parameter for the initial
  system prompt; the directive-only form (content: [] with output_config) is accepted at any position"},"request_id":"req_011Ce9Bu9MLWNsCnaBVHqXhs"}.
  Received Model Group=claude-opus-5
  Available Model Group Fallbacks=None

3. Why a first turn trips it: Claude Code sends its SessionStart hook output as an in-sequence role: system row right after the first user turn, so the write-back treats the request as one that preserves system rows and emits the guardrail-masked system prompt into messages[0]. The proxy's outgoing request in the --detailed_debug log confirms it: messages[0] is {'role': 'system', ...} carrying the masked Claude Code system prompt, while the top-level system list still holds the unmasked original (the raw home-directory path appears once)

Leading system row in messages

1. Run

curl -s -w "\nHTTP %{http_code}\n" http://localhost:56417/v1/messages -H "Authorization: Bearer sk-lit5696-test" -H "Content-Type: application/json" -d @req_leading.json

2. Observed

{"error":{"message":"{\"type\":\"error\",\"error\":{\"type\":\"invalid_request_error\",\"message\":\"messages.0: use the top-level 'system' parameter for the initial system prompt; the directive-only form (content: [] with output_config) is accepted at any position\"},\"request_id\":\"req_011Ce99kZ2PiEqZ7ELyLthnd\"}. Received Model Group=claude-opus-5\nAvailable Model Group Fallbacks=None","type":"None","param":"None","code":"400"}}
HTTP 400

Mid-turn system after a user turn

1. Run

curl -s -w "\nHTTP %{http_code}\n" http://localhost:56901/v1/messages -H "Authorization: Bearer sk-lit5696-test" -H "Content-Type: application/json" -d @req_midturn_valid.json

2. Observed

{"error":{"message":"{\"type\":\"error\",\"error\":{\"type\":\"invalid_request_error\",\"message\":\"messages.0: use the top-level 'system' parameter for the initial system prompt; the directive-only form (content: [] with output_config) is accepted at any position\"},\"request_id\":\"req_011Ce9A1RnVLF49tEyCNtrQG\"}. Received Model Group=claude-opus-5\nAvailable Model Group Fallbacks=None","type":"None","param":"None","code":"400"}}
HTTP 400

Masked system prompt delivery

1. Run

curl -s -w "\nHTTP %{http_code}\n" http://localhost:56417/v1/messages -H "Authorization: Bearer sk-lit5696-test" -H "Content-Type: application/json" -d @req_echo.json

2. Observed: HTTP 200 but the answer echoes the original unmasked address, so the guardrail's masking was silently lost before reaching Anthropic

{"model":"claude-opus-5","id":"msg_011Ce99m92Y6uNQMpXkVHgJS","type":"message","role":"assistant","content":[{"type":"text","text":"bob@example.com"}],"stop_reason":"end_turn",...}
HTTP 200

Mid-turn system after an assistant turn

1. Run

curl -s -w "\nHTTP %{http_code}\n" http://localhost:56417/v1/messages -H "Authorization: Bearer sk-lit5696-test" -H "Content-Type: application/json" -d @req_midturn.json

2. Observed: the same misleading messages.0 error even though the client's system row sits at index 2

{"error":{"message":"{\"type\":\"error\",\"error\":{\"type\":\"invalid_request_error\",\"message\":\"messages.0: use the top-level 'system' parameter for the initial system prompt; the directive-only form (content: [] with output_config) is accepted at any position\"},\"request_id\":\"req_011Ce99mc9wHc5PCY6Yz5G5y\"}. Received Model Group=claude-opus-5\nAvailable Model Group Fallbacks=None","type":"None","param":"None","code":"400"}}
HTTP 400

Plain top-level system

1. Run

curl -s -w "\nHTTP %{http_code}\n" http://localhost:56417/v1/messages -H "Authorization: Bearer sk-lit5696-test" -H "Content-Type: application/json" -d @req_plain.json

2. Observed

{"model":"claude-opus-5",...,"content":[{"type":"text","text":"Hello."}],"stop_reason":"end_turn",...}
HTTP 200

Same flow on /v1/chat/completions

1. Run

curl -s -w "\nHTTP %{http_code}\n" http://localhost:56417/v1/chat/completions -H "Authorization: Bearer sk-lit5696-test" -H "Content-Type: application/json" -d @req_chat.json

2. Observed

{"id":"chatcmpl-6d7697f0-570a-48e1-9f75-623f871b8202",...,"choices":[{"finish_reason":"stop","index":0,"message":{"content":"Hello.","role":"assistant",...}}],...}
HTTP 200

After (0cbec3f)

Claude Code first turn (reporter's client)

1. Same TUI flow against the proxy booted at 0cbec3f on port 34012

tmux new-session -d -s lit5696_tip -x 160 -y 45 -c ./work "env -u ANTHROPIC_API_KEY ANTHROPIC_BASE_URL=http://localhost:34012 ANTHROPIC_AUTH_TOKEN=sk-lit5696-test ANTHROPIC_MODEL=claude-opus-5 claude"
tmux send-keys -t lit5696_tip "test" Enter
tmux capture-pane -p -t lit5696_tip

2. Observed on screen (pane trimmed the same way): a normal answer

❯ test
⏺ Test received, everything is working.
✻ Sautéed for 7s

3. The outgoing Anthropic request now starts with the user turn, Claude Code's SessionStart system row stays in place after it, and the top-level system list carries the masked prompt (/Users/<USER> twice, raw home-directory path zero times)

The curl cases below were captured at a972f17 and carry over to 0cbec3f unchanged (annotation-only commit)

Leading system row in messages

1. Run

curl -s -w "\nHTTP %{http_code}\n" http://localhost:57431/v1/messages -H "Authorization: Bearer sk-lit5696-test" -H "Content-Type: application/json" -d @req_leading.json

2. Observed: the leading row is folded into Anthropic's top-level system parameter

{"model":"claude-opus-5","id":"msg_011Ce99mVdH4NxQvttctbDtn","type":"message","role":"assistant","content":[{"type":"text","text":"Hello."}],"stop_reason":"end_turn",...}
HTTP 200

Mid-turn system after a user turn

1. Run

curl -s -w "\nHTTP %{http_code}\n" http://localhost:57903/v1/messages -H "Authorization: Bearer sk-lit5696-test" -H "Content-Type: application/json" -d @req_midturn_valid.json

2. Observed: 200 and the answer is uppercase, so the mid-turn directive stayed in its original position and reached the model

{"model":"claude-opus-5",...,"content":[{"type":"thinking",...},{"type":"text","text":"DONE"}],"stop_reason":"end_turn",...}
HTTP 200

Masked system prompt delivery

1. Run the identical request; at max_tokens 64 claude-opus-5 spent the whole budget thinking and returned 200 with no visible text, so the readable rerun below raises only max_tokens to 1024

jq '.max_tokens = 1024' req_echo.json > req_echo_1024.json
curl -s -w "\nHTTP %{http_code}\n" http://localhost:57431/v1/messages -H "Authorization: Bearer sk-lit5696-test" -H "Content-Type: application/json" -d @req_echo_1024.json

2. Observed: the answer echoes the masked placeholder, and bob@example.com appears in no response from this leg, so the guardrail-masked system prompt is what Anthropic received

{"model":"claude-opus-5","id":"msg_011Ce99rttQFDZwTpcs3PUt7",...,"content":[{"type":"text","text":"<EMAIL>\n\nThat's the literal string in my instructions — it's a placeholder, not a working address. You'd need the actual address from whoever set this up."}],"stop_reason":"end_turn",...}
HTTP 200

Mid-turn system after an assistant turn

1. Run

curl -s -w "\nHTTP %{http_code}\n" http://localhost:57431/v1/messages -H "Authorization: Bearer sk-lit5696-test" -H "Content-Type: application/json" -d @req_midturn.json

2. Observed: Anthropic's own 400 at the row's true index (its live validator only accepts a content-carrying mid-turn system row that follows a user turn and precedes an assistant turn or ends the array)

{"error":{"message":"{\"type\":\"error\",\"error\":{\"type\":\"invalid_request_error\",\"message\":\"messages.2: role 'system' must follow a 'user' message or an 'assistant' message ending in a server tool result; ...\"},...}. Received Model Group=claude-opus-5\n...","code":"400"}}
HTTP 400

3. Parity check: the same payload family sent straight to api.anthropic.com returns the identical error at the identical index, shown here with req_midturn_user.json (system row between two user turns, rejected at messages.3 by the proxy and by raw Anthropic alike)

curl -s -w "\nHTTP %{http_code}\n" http://localhost:57431/v1/messages -H "Authorization: Bearer sk-lit5696-test" -H "Content-Type: application/json" -d @req_midturn_user.json
{"error":{"message":"{\"type\":\"error\",\"error\":{\"type\":\"invalid_request_error\",\"message\":\"messages.3: role 'system' must precede an 'assistant' message or end the array; ...\"},...}. Received Model Group=claude-opus-5\n...","code":"400"}}
HTTP 400

curl -s -w "\nHTTP %{http_code}\n" https://api.anthropic.com/v1/messages -H "x-api-key: $ANTHROPIC_API_KEY" -H "anthropic-version: 2023-06-01" -H "Content-Type: application/json" -d @req_midturn_user.json
{"type":"error","error":{"type":"invalid_request_error","message":"messages.3: role 'system' must precede an 'assistant' message or end the array; the directive-only form (content: [] with output_config) is accepted at any position"},"request_id":"req_011Ce99ykkSyGirtknDon8d7"}
HTTP 400

Plain top-level system

1. Run

curl -s -w "\nHTTP %{http_code}\n" http://localhost:57431/v1/messages -H "Authorization: Bearer sk-lit5696-test" -H "Content-Type: application/json" -d @req_plain.json

2. Observed

{"model":"claude-opus-5",...,"content":[{"type":"text","text":"Hello."}],"stop_reason":"end_turn",...}
HTTP 200

Same flow on /v1/chat/completions

1. Run

curl -s -w "\nHTTP %{http_code}\n" http://localhost:57431/v1/chat/completions -H "Authorization: Bearer sk-lit5696-test" -H "Content-Type: application/json" -d @req_chat.json

2. Observed

{"id":"chatcmpl-176b9c38-dfb5-4525-be7e-4df1e6c3a2f1",...,"choices":[{"finish_reason":"stop","index":0,"message":{"content":"Hello.","role":"assistant",...}}],...}
HTTP 200

Cross-provider check: the same handler serves every model behind /v1/messages, so the same config was rerun with bedrock/us.anthropic.claude-sonnet-5 at base 973329e and tip 0cbec3f (real Bedrock calls). req_leading, req_midturn_valid, and req_plain return 200 on both sides (Bedrock's own transform already hoists in-sequence system rows), and the masked-echo probe flips from bob@example.com at base to <EMAIL> at tip, so the PII leak was cross-provider and is closed cross-provider

Notes observed during QA, none caused by this PR and all left alone by it:

• Anthropic also rejects system rows placed before user turns
• Mid-turn system after plain assistant 400s; equals no-guardrail behavior
• claude-opus-5 thinks by default; tiny max_tokens yields empty text
• Skills injection concatenates a str onto any list-shaped system; pre-existing

Type

🐛 Bug Fix

Caveats (if any)

• Leading system rows now fold into the system param, never messages
• A rewrite that drops the hoisted prompt replaces the system param

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

• 0cbec3f passes /live-pr-risk

[greptile-apps]

Greptile Summary

The PR updates Anthropic guardrail write-back so leading system rows are folded into the provider’s top-level system parameter while mid-turn rows retain their position.

• Preserves guardrail-modified top-level prompts.
• Adds regression coverage for leading, mid-turn, masked, and all-system rewrite cases.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
litellm/llms/anthropic/chat/guardrail_translation/handler.py	Adds top-level folding for leading system rows and keeps the existing mid-turn and tool-exchange conversion behavior.
tests/test_litellm/llms/anthropic/chat/guardrail_translation/test_anthropic_guardrail_handler.py	Extends guardrail rewrite coverage across masked top-level prompts, leading rows, mid-turn rows, and system-only outputs.

_{Reviews (2): Last reviewed commit: "refactor(anthropic): drop bare generics ..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 0cbec3f. Configure here.}

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_lit5696_system_hoist_writeback (0cbec3f) with litellm_internal_staging (2bc87ec)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (50e7131) during the generation of this report, so 2bc87ec was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[tin-berri]

LGTM — critical regression fix, reproduced live with the actual Claude Code TUI plus curl edge cases (leading system row, mid-turn after user/assistant, masked-prompt leak), and verified for error parity against raw api.anthropic.com. Good scope: leading system rows fold into the top-level system param while mid-turn directives keep their position, and the guardrail-masked prompt now actually reaches Anthropic instead of leaking the unmasked original. Cross-provider Bedrock check confirms the PII leak was closed there too.