fix(transformation): merge consecutive system and developer messages for non-OpenAI providers and Responses API

[sowmy86]

TLDR

Problem this solves:

• Responses API requests with instructions plus developer input produced several system messages
• Two native system messages also reached backends unmerged
• A developer message placed after the first user turn (Codex CLI's per-turn permissions item, a re-injected instruction after compaction, a mid-conversation policy update) was translated to a system message in place, so it landed mid-conversation
• System-first chat templates (Fireworks qwen3.8, Qwen, Ollama, DeepSeek) reject any of these with 400 System message must be at the beginning.

How it solves it:

• Developer role translation now folds consecutive system messages into one
• Developer messages that arrive after the leading system block are hoisted into that block first, so they merge into the single leading system message; client-authored mid-conversation system messages stay where the client put them
• The Responses bridge merges instructions into a leading system or developer item
• String contents join with a blank line, block lists concatenate

Trade-off called out for reviewers: a hoisted developer instruction is applied from the start of the conversation rather than at the position the client placed it. For system-first templates the alternative is a hard 400, and for templates that accept later system messages the instruction content is unchanged, only its position moves. Providers that receive the developer role natively (OpenAI o-series and gpt-5 family) are untouched

User Flow

Before: a Codex CLI session pointed at the gateway with a Fireworks qwen3.8 deployment dies on its first turn, and a developer instruction placed after the first input item fails the same way from any Responses API client

1. They configure Codex CLI to use the gateway (base_url = https://litellm-domain/v1, wire_api = "responses", model fireworks-qwen3p8) and start a session in a repo
2. They type What is the capital of France? Answer in one word. and Codex sends POST https://litellm-domain/v1/responses with instructions, a developer item carrying its permissions text, a user item carrying AGENTS.md, and a user item carrying the prompt
3. The pane shows a red 400 from the gateway, jinja template rendering failed. System message must be at the beginning., and no reply
4. Their own Responses API client sends POST https://litellm-domain/v1/responses with input of a user item, then a developer item Answer with exactly one word., then the user question
5. The gateway answers the same 400 and the client gets no output

After: the same session answers on its first turn, and a developer instruction placed after the first input item is honored

1. They configure Codex CLI to use the gateway (base_url = https://litellm-domain/v1, wire_api = "responses", model fireworks-qwen3p8) and start a session in a repo
2. They type What is the capital of France? Answer in one word. and Codex sends POST https://litellm-domain/v1/responses with instructions, a developer item carrying its permissions text, a user item carrying AGENTS.md, and a user item carrying the prompt
3. The pane shows Paris and the session keeps going
4. Their own Responses API client sends POST https://litellm-domain/v1/responses with input of a user item, then a developer item Answer with exactly one word., then the user question
5. The gateway answers 200 with a single-word output text, Paris

Relevant issues

Fixes #26879

Linear ticket

Resolves LIT-7019

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Shared setup, identical for both legs. Two proxies from two worktrees, each with --num_workers 2, hitting the real Fireworks API (real $$$)

model_list:
  - model_name: fireworks-qwen3p8
    litellm_params:
      model: fireworks_ai/accounts/fireworks/models/qwen3p8-2p4t-a95b
      api_key: os.environ/FIREWORKS_API_KEY
      additional_drop_params: ["client_metadata"]

general_settings:
  master_key: sk-lit7019

# Before: worktree at the merge base 4990f06acc
python litellm/proxy/proxy_cli.py --config repro_config.yaml --port 21943 --num_workers 2 --detailed_debug
# After: worktree at the PR tip 25186b1e7b
python litellm/proxy/proxy_cli.py --config repro_config.yaml --port 29240 --num_workers 2 --detailed_debug

Codex CLI 0.145.0 config.toml (only base_url differs between legs):

model = "fireworks-qwen3p8"
model_provider = "litellm"

[model_providers.litellm]
name = "LiteLLM"
base_url = "http://localhost:29240/v1"
env_key = "LITELLM_QA_KEY"
wire_api = "responses"

additional_drop_params: ["client_metadata"] is unrelated to this PR: Codex sends client_metadata on every request and Fireworks rejects it (tracked in #28539 and #36268)

Before (4990f06)

Codex CLI first turn

1. LITELLM_QA_KEY=sk-lit7019 codex in a trusted repo, type What is the capital of France? Answer with exactly one word. and press Enter. Codex sends instructions, a developer item with its permissions text, a user item with AGENTS.md, and the prompt
2. Observed: a red 400 in the pane and no answer

/v1/responses, developer item after the first input item (the customer's captured shape)

1. Run

   curl -sS http://localhost:21943/v1/responses -H 'Authorization: Bearer sk-lit7019' -H 'content-type: application/json' -d '{"model":"fireworks-qwen3p8","input":[{"role":"user","content":"Hi there"},{"role":"developer","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 400
   error: litellm.BadRequestError: Fireworks_aiException - {"error":{"object":"error","type":"invalid_request_error","code":"invalid_request_error","message":"jinja template rendering failed. System message must be at the beginning."}}. Received Model Group=fireworks-qwen3p8
   

/v1/responses, instructions plus a leading developer item (Codex first turn shape)

1. Run

   curl -sS http://localhost:21943/v1/responses -H 'Authorization: Bearer sk-lit7019' -H 'content-type: application/json' -d '{"model":"fireworks-qwen3p8","instructions":"You are a terse assistant.","input":[{"role":"developer","content":[{"type":"input_text","text":"Answer with exactly one word."}]},{"role":"user","content":[{"type":"input_text","text":"What is the capital of France?"}]}]}'
   

2. Observed

   HTTP 400
   error: litellm.BadRequestError: Fireworks_aiException - {"error":{"object":"error","type":"invalid_request_error","code":"invalid_request_error","message":"jinja template rendering failed. System message must be at the beginning."}}. Received Model Group=fireworks-qwen3p8
   

/v1/chat/completions, developer message after the first user turn

1. Run

   curl -sS http://localhost:21943/v1/chat/completions -H 'Authorization: Bearer sk-lit7019' -H 'content-type: application/json' -d '{"model":"fireworks-qwen3p8","messages":[{"role":"system","content":"You are a terse assistant."},{"role":"user","content":"Hi there"},{"role":"assistant","content":"Hello! How can I help?"},{"role":"developer","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 400
   error: litellm.BadRequestError: Fireworks_aiException - {"error":{"object":"error","type":"invalid_request_error","code":"invalid_request_error","message":"jinja template rendering failed. System message must be at the beginning."}}. Received Model Group=fireworks-qwen3p8
   

/v1/messages, system prompt with a multi-turn conversation (control, unaffected path)

1. Run

   curl -sS http://localhost:21943/v1/messages -H 'Authorization: Bearer sk-lit7019' -H 'content-type: application/json' -d '{"model":"fireworks-qwen3p8","max_tokens":64,"system":"Answer with exactly one word.","messages":[{"role":"user","content":"Hi there"},{"role":"assistant","content":"Hello!"},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 200
   assistant: Paris
   

After (25186b1)

Codex CLI first turn

1. Same Codex session against port 29240, same prompt
2. Observed: Paris

3. /permissions, pick Approve for me, then type And of Germany? Same rule, one word. Codex now sends the customer's exact shape, a developer item after the first user turn: input roles developer, user, user, assistant, developer, user

4. Observed: Berlin, and the proxy log shows the hoist doing the work

   19:17:59 - LiteLLM:DEBUG: base_utils.py:287 - Hoisting 1 developer message(s) into the leading system block for non-OpenAI providers.
   

/v1/responses, developer item after the first input item (the customer's captured shape)

1. Run

   curl -sS http://localhost:29240/v1/responses -H 'Authorization: Bearer sk-lit7019' -H 'content-type: application/json' -d '{"model":"fireworks-qwen3p8","input":[{"role":"user","content":"Hi there"},{"role":"developer","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 200
   output_text: Paris
   

/v1/responses, instructions plus a leading developer item (Codex first turn shape)

1. Run

   curl -sS http://localhost:29240/v1/responses -H 'Authorization: Bearer sk-lit7019' -H 'content-type: application/json' -d '{"model":"fireworks-qwen3p8","instructions":"You are a terse assistant.","input":[{"role":"developer","content":[{"type":"input_text","text":"Answer with exactly one word."}]},{"role":"user","content":[{"type":"input_text","text":"What is the capital of France?"}]}]}'
   

2. Observed

   HTTP 200
   output_text: Paris
   

/v1/chat/completions, developer message after the first user turn

1. Run

   curl -sS http://localhost:29240/v1/chat/completions -H 'Authorization: Bearer sk-lit7019' -H 'content-type: application/json' -d '{"model":"fireworks-qwen3p8","messages":[{"role":"system","content":"You are a terse assistant."},{"role":"user","content":"Hi there"},{"role":"assistant","content":"Hello! How can I help?"},{"role":"developer","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 200
   assistant: Paris
   

/v1/messages, system prompt with a multi-turn conversation (control, unaffected path)

1. Run

   curl -sS http://localhost:29240/v1/messages -H 'Authorization: Bearer sk-lit7019' -H 'content-type: application/json' -d '{"model":"fireworks-qwen3p8","max_tokens":64,"system":"Answer with exactly one word.","messages":[{"role":"user","content":"Hi there"},{"role":"assistant","content":"Hello!"},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 200
   assistant: Paris
   

Type

🐛 Bug Fix

Caveats (if any)

Low

• Hoisted developer instructions apply from conversation start, not their original position
• Merging applies to all non-OpenAI providers, including ones tolerating multiple system messages
• When both merged messages set cache_control, the later marker wins, matching breakpoint semantics
• Billing metadata system blocks are never merged, keeping provider-side stripping intact
• Client-authored mid-conversation system messages still stay where they are
• Native Fireworks Responses path in feat(fireworks_ai): add native Responses API config #39826 needs the same hoist once it lands
• Codex client_metadata still needs additional_drop_params (Custom_openaiException - AsyncCompletions.create() got an unexpected keyword argument 'client_metadata' #28539, fix(responses): stop the chat completions bridge from leaking unknown request fields #36268)

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing sowmy86:fix-merge-consecutive-system-developer-messages (25186b1) with litellm_internal_staging (2151dcb)}

[greptile-apps]

Greptile Summary

This PR merges consecutive system and translated developer messages for non-OpenAI providers and combines Responses API instructions with leading system/developer input.

• Preserves later message metadata, including cache_control, during merges.
• Keeps billing metadata messages separate so downstream provider transformations can strip them.
• Adds regression coverage for string and block content, metadata precedence, billing metadata, and Responses API inputs.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the previously reported loss of later cache_control metadata is fixed by applying the later message’s fields during merging and is covered by regression tests.

Important Files Changed

Filename	Overview
litellm/llms/base_llm/base_utils.py	Adds shared content-merging helpers and folds consecutive system/developer messages while preserving the later message’s metadata.
litellm/responses/litellm_completion_transformation/transformation.py	Merges Responses API instructions into a leading system or developer input while retaining the original message shape.
tests/test_litellm/llms/base_llm/test_base_utils.py	Covers content combinations, cache metadata precedence, billing metadata boundaries, and consecutive-message behavior.
tests/test_litellm/responses/litellm_completion_transformation/test_litellm_completion_responses.py	Covers instruction merging with leading system/developer input and fallback behavior for empty or user-only input.

_{Reviews (2): Last reviewed commit: "fix(base_utils): keep later cache_contro..." | Re-trigger Greptile}

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 2 · PR risk: 0/10

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[sowmy86]

@greptileai please re-review, both findings are addressed in 40fab55 with regression tests and the PR description is updated

[CLAassistant]

All committers have signed the CLA.

[sowmy86]

Hi maintainers 👋 — this is ready for review. Rebased onto the latest litellm_internal_staging, no conflicts, and all checks are green (including OSV Scan and 100% patch coverage).

It fixes #26879: consecutive system/developer messages are now merged into one before reaching non-OpenAI providers, on both the Responses API bridge and the chat completions path. 15 regression tests added.

Could someone with write access take a look and approve? Happy to address any feedback. Thanks!

[sowmy86]

Hi maintainers 👋 this is ready for review: rebased on the latest litellm_internal_staging, all checks green, and all review-bot threads resolved.

It fixes #26879 by merging consecutive system/developer messages before they reach non-OpenAI providers, on both the Responses API bridge and the chat completions path. 15 regression tests added.

Note: the last commit also drops a duplicate parameter in the Bedrock KB hook test that #39474 introduced on staging, since it was failing the required lint job for every PR. Happy to split it out if you'd rather land that separately.

Could someone with write access take a look? Thanks!

[sowmy86]

Hi maintainers 👋 — this PR is ready for review. All 33 required checks pass, it's rebased on the latest litellm_internal_staging with no conflicts, and all review-bot threads are resolved.

The bug (#26879): providers that accept only a single system prompt (Anthropic, Bedrock, Gemini, etc.) reject requests when LiteLLM sends more than one system message. This happens in two places:

• The Responses API bridge emits instructions as a system message and then a separate one for a developer input item.
• The chat completions path passes consecutive system messages through untouched.

The fix:map_developer_role_to_system_role now translates developer → system and folds consecutive system messages into one, in a single linear pass. Block-level cache_control and Anthropic billing-metadata boundaries are preserved. The Responses bridge merges instructions into a leading system/developer message instead of emitting a second one. 15 regression tests added, 100% patch coverage.

This also covers the chat completions case, which #26884 and #26888 don't address.

Note on OSV Scan: the red mark is the new gitpython advisories (fix is 3.1.59), which affect every PR on staging today and are being handled in #39553. It's not a required check and this PR doesn't touch the lockfile.

Could someone with write access take a look? Happy to address any feedback. Thanks!

[mateo-berri]

Superseded by #39852, an internal copy with your commits intact, so CircleCI's provider suites can run on a litellm_ branch. Thanks for the fix!

[sowmy86]

Superseded by #39852, an internal copy with your commits intact, so CircleCI's provider suites can run on a litellm_ branch. Thanks for the fix!

Thanks for carrying it forward, glad it was useful. Happy to help with anything on #39852.