fix(transformation): merge system and developer messages into one leading system message (internal copy of #39282)

[mateo-berri]

Internal copy of #39282 at head 25186b1, commits and authors intact, so CircleCI's provider suites run on a litellm_ branch. The follow-up commits on top narrow the hoist to OpenAI-compatible backends, answer the review comments, merge each run of system messages in a single pass, keep a closing developer message in place, and share the item-picking rule with the native Fireworks Responses route that #39826 and #40268 landed on staging

TLDR

Problem this solves:

• A developer message placed after the first user turn (a per-turn permissions item, a re-injected instruction after compaction, a mid-conversation policy update) was translated to a system message in place, so it landed mid-conversation
• Responses API requests with instructions plus developer input produced several system messages on the chat completions bridge
• System-first chat templates (Fireworks qwen3.8, Qwen, DeepSeek, and other OpenAI-compatible backends) reject both with 400 System message must be at the beginning.

How it solves it:

• Developer role translation is now a method on the provider config, so the behavior depends on the backend
• OpenAI-compatible backends (Fireworks, DeepSeek, Together, vLLM, Groq, Azure AI Foundry, openai/ with a custom api_base, every config that inherits OpenAIGPTConfig or OpenAIConfig) hoist developer messages that arrive after the leading system block into that block and fold the block into one leading system message. A developer message that closes the conversation right after an assistant turn stays where it is, translated in place as before, so the request never ends on the assistant's turn, and on a system-first template that closing shape still gets the same 400 it got before this PR. Client-authored mid-conversation system messages stay where the client put them
• The native Fireworks Responses route already folds instructions, the leading system and developer items, and every later developer item into top-level instructions since fix(fireworks_ai): fold instructions and developer items into one leading system message on the Responses path #40268 (in this PR's merge base). This PR moves that route's index-picking helper into base_utils as hoisted_instruction_indices, so the chat hoist and the native route share one closing-item rule instead of two private copies
• litellm_proxy/ hoists the same way, so a developer message reaches the downstream LiteLLM proxy already folded into the leading system message whichever backend it routes to (and whichever LiteLLM version it runs)
• A system message with null or missing content inside a folded run is dropped instead of raising, and a message-level cache_control on a list-content member lands on that member's last block so the breakpoint survives the fold
• OpenAI's own endpoint and Azure OpenAI keep the in-place translation they had, and the o-series keeps passing developer through natively
• Providers with their own message format (Anthropic, Bedrock Converse, Gemini, Ollama) are untouched: the wire request is byte-identical before and after. The OpenAI-compatible routes those providers also expose (bedrock/openai/..., vertex_ai/meta/... and the other Vertex model-garden configs) inherit the hoist like every other config built on OpenAIGPTConfig
• Two plain strings join with a blank line. Anything carrying cache_control or Anthropic billing metadata is folded as its own text block, so each breakpoint stays on its own block and provider-side stripping still finds the metadata

Trade-off called out for reviewers: on OpenAI-compatible backends a hoisted developer instruction is applied from the start of the conversation rather than at the position the client placed it. For system-first templates the alternative is a hard 400, and for templates that accept later system messages the instruction content is unchanged, only its position moves

User Flow

Before: a Responses API client that places a developer item after the first input item gets a 400 from a Fireworks qwen3.8 deployment on the chat completions bridge and through the reporter's custom_openai/ prefix, the same messages fail on /v1/chat/completions, and a Codex CLI session on the bridge deployment never gets an answer

1. They add fireworks-qwen3p8-bridge (Fireworks qwen3.8 with use_chat_completions_api: true) and qwen3p8-custom-openai (the same model through custom_openai/ with the Fireworks base URL) to the gateway config and point their agent at the gateway
2. Their Responses API client sends POST https://litellm-domain/v1/responses to fireworks-qwen3p8-bridge with input of a user item Hi there, then a developer item Answer with exactly one word., then the user question What is the capital of France?
3. The gateway answers 400 jinja template rendering failed. System message must be at the beginning. and the client gets no output
4. They send the same request to qwen3p8-custom-openai
5. The gateway answers the same 400
6. They send the same three messages to POST https://litellm-domain/v1/chat/completions on fireworks-qwen3p8-bridge
7. The gateway answers the same 400
8. They configure Codex CLI to use the gateway (base_url = https://litellm-domain/v1, wire_api = "responses", model fireworks-qwen3p8-bridge), start a session in a repo, and type What is the capital of France? Answer with exactly one word.
9. The pane shows the gateway's 400 in red, System message must be at the beginning., and no reply; the follow-up turn And of Germany? Answer with exactly one word. fails the same way

After: the same requests are answered with the instruction honored, and the Codex session on the bridge deployment answers both turns

1. They add fireworks-qwen3p8-bridge (Fireworks qwen3.8 with use_chat_completions_api: true) and qwen3p8-custom-openai (the same model through custom_openai/ with the Fireworks base URL) to the gateway config and point their agent at the gateway
2. Their Responses API client sends POST https://litellm-domain/v1/responses to fireworks-qwen3p8-bridge with input of a user item Hi there, then a developer item Answer with exactly one word., then the user question What is the capital of France?
3. The gateway answers 200 with a single output text, Paris
4. They send the same request to qwen3p8-custom-openai
5. The gateway answers 200 with Paris
6. They send the same three messages to POST https://litellm-domain/v1/chat/completions on fireworks-qwen3p8-bridge
7. The gateway answers 200 with Paris
8. They configure Codex CLI to use the gateway (base_url = https://litellm-domain/v1, wire_api = "responses", model fireworks-qwen3p8-bridge), start a session in a repo, and type What is the capital of France? Answer with exactly one word.
9. The pane shows Paris, and the follow-up turn And of Germany? Answer with exactly one word. shows Berlin

Relevant issues

Fixes #26879 (the stale bot closed it before a fix landed)

Supersedes #26884

Related: #39826 added the native Fireworks Responses route on staging and #40268 folds developer items into instructions there; both are in this PR's merge base, which is why the native route answers 200 on both legs below

Linear ticket

Resolves LIT-7019

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Shared setup, identical for both legs: two proxies, one at the merge base db7ca65 (staging head at the time, port 57770) and one at the PR tip 8b49a64 (port 48910), each booted with --num_workers 2 and no database, hitting the real Fireworks and OpenAI APIs (real $$$). fireworks-qwen3p8 is the native Fireworks Responses route (the default for fireworks_ai/ models since #39826), fireworks-qwen3p8-bridge is the same model through the chat completions bridge, fireworks-qwen3p8-openai-compat is the same model through openai/ with the Fireworks base URL, qwen3p8-custom-openai is the issue reporter's prefix, qwen3p8-hosted-vllm and qwen3p8-hosted-vllm-native-responses stand in for the second commenter's vLLM stack (Fireworks renders the same Qwen chat template a vLLM server would and rejects a later system message the same way), and gpt-5.4-mini is the control that must keep its current shape

model_list:
  - model_name: fireworks-qwen3p8
    litellm_params:
      model: fireworks_ai/accounts/fireworks/models/qwen3p8-2p4t-a95b
      api_key: os.environ/FIREWORKS_API_KEY
      additional_drop_params: ["client_metadata"]
  - model_name: fireworks-qwen3p8-bridge
    litellm_params:
      model: fireworks_ai/accounts/fireworks/models/qwen3p8-2p4t-a95b
      api_key: os.environ/FIREWORKS_API_KEY
      use_chat_completions_api: true
      drop_params: true
      additional_drop_params: ["client_metadata"]
  - model_name: fireworks-qwen3p8-openai-compat
    litellm_params:
      model: openai/accounts/fireworks/models/qwen3p8-2p4t-a95b
      api_base: https://api.fireworks.ai/inference/v1
      api_key: os.environ/FIREWORKS_API_KEY
  - model_name: gpt-5.4-mini
    litellm_params:
      model: openai/gpt-5.4-mini
      api_key: os.environ/OPENAI_API_KEY
  - model_name: qwen3p8-custom-openai
    litellm_params:
      model: custom_openai/accounts/fireworks/models/qwen3p8-2p4t-a95b
      api_base: https://api.fireworks.ai/inference/v1
      api_key: os.environ/FIREWORKS_API_KEY
  - model_name: qwen3p8-hosted-vllm
    litellm_params:
      model: hosted_vllm/accounts/fireworks/models/qwen3p8-2p4t-a95b
      api_base: https://api.fireworks.ai/inference/v1
      api_key: os.environ/FIREWORKS_API_KEY
      use_chat_completions_api: true
      drop_params: true
  - model_name: qwen3p8-hosted-vllm-native-responses
    litellm_params:
      model: hosted_vllm/accounts/fireworks/models/qwen3p8-2p4t-a95b
      api_base: https://api.fireworks.ai/inference/v1
      api_key: os.environ/FIREWORKS_API_KEY

general_settings:
  master_key: sk-1234

# Before: checkout at the merge base db7ca65b69
PYTHONPATH=$PWD python litellm/proxy/proxy_cli.py --config lit7019_config.yaml --port 57770 --num_workers 2 --detailed_debug
# After: worktree at the PR tip 8b49a6438c
PYTHONPATH=$PWD python litellm/proxy/proxy_cli.py --config lit7019_config.yaml --port 48910 --num_workers 2 --detailed_debug

Both proxies were launched through a short shim that loads the QA .env and unsets DATABASE_URL before handing those flags to proxy_cli.run_server; it holds no LiteLLM logic

Codex CLI 0.145.0 config.toml (base_url differs between legs, model between the two Codex cases):

model = "fireworks-qwen3p8-bridge"
model_provider = "litellm"

[model_providers.litellm]
name = "LiteLLM"
base_url = "http://localhost:48910/v1"
env_key = "LITELLM_QA_KEY"
wire_api = "responses"

additional_drop_params: ["client_metadata"] and the bridge aliases' drop_params: true are unrelated to this PR and identical on both legs: Codex sends client_metadata on every request and Fireworks rejects it (tracked in #28539 and #36268), and the bridge rejects Codex's tool_choice and reasoning as unsupported for this model without drop_params

Every curl below ran from one script in the order shown, once per proxy. The "Observed" block is what that script printed: the HTTP status, then the response's output text (output_text for /v1/responses, assistant for /v1/chat/completions and /v1/messages) or its error.message on a failure. The "Upstream request" step is read from the proxy's --detailed_debug log so the reviewer can see what the provider received; the verdict rests on the HTTP responses alone

Before (db7ca65)

/v1/responses, developer item after the first input item, native Fireworks Responses route

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:57770/v1/responses -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"fireworks-qwen3p8","input":[{"role":"user","content":"Hi there"},{"role":"developer","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 200
   output_text: ['Paris']
   

3. Upstream request to POST https://api.fireworks.ai/inference/v1/responses: input roles ['user', 'user'], instructions 'Answer with exactly one word.'

/v1/responses, same input through the chat completions bridge

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:57770/v1/responses -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"fireworks-qwen3p8-bridge","input":[{"role":"user","content":"Hi there"},{"role":"developer","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 400
   error: litellm.BadRequestError: Fireworks_aiException - {"error":{"object":"error","type":"invalid_request_error","code":"invalid_request_error","message":"jinja template rendering failed. System message must be at the beginning."}}. Received Model Group=fireworks-qwen3p8-bridge
   Available Model Group Fallb
   

3. Upstream request to POST https://api.fireworks.ai/inference/v1/chat/completions: messages roles ['user', 'system', 'user']

/v1/responses, same input to OpenAI (control)

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:57770/v1/responses -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"gpt-5.4-mini","input":[{"role":"user","content":"Hi there"},{"role":"developer","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 200
   output_text: ['Paris']
   

3. Upstream request to POST https://api.openai.com/v1/responses: input roles ['user', 'developer', 'user']

/v1/responses, same input through the reporter's custom_openai/ prefix (chat completions bridge)

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:57770/v1/responses -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"qwen3p8-custom-openai","input":[{"role":"user","content":"Hi there"},{"role":"developer","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 400
   error: litellm.BadRequestError: Custom_openaiException - jinja template rendering failed. System message must be at the beginning.. Received Model Group=qwen3p8-custom-openai
   Available Model Group Fallbacks=None
   

3. Upstream request to POST https://api.fireworks.ai/inference/v1/chat/completions: messages roles ['user', 'system', 'user']

/v1/responses, same input through hosted_vllm/ on the chat completions bridge

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:57770/v1/responses -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"qwen3p8-hosted-vllm","input":[{"role":"user","content":"Hi there"},{"role":"developer","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 400
   error: litellm.BadRequestError: Hosted_vllmException - {"error":{"object":"error","type":"invalid_request_error","code":"invalid_request_error","message":"jinja template rendering failed. System message must be at the beginning."}}. Received Model Group=qwen3p8-hosted-vllm
   Available Model Group Fallbacks=N
   

3. Upstream request to POST https://api.fireworks.ai/inference/v1/chat/completions: messages roles ['user', 'system', 'user']

/v1/responses, same input through hosted_vllm/ on its native Responses route (pre-existing, this PR leaves it alone)

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:57770/v1/responses -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"qwen3p8-hosted-vllm-native-responses","input":[{"role":"user","content":"Hi there"},{"role":"developer","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 400
   error: litellm.BadRequestError: Hosted_vllmException - {"error":{"type":"invalid_request_error","code":"invalid_request_error","message":"jinja template rendering failed. System message must be at the beginning."}}. Received Model Group=qwen3p8-hosted-vllm-native-responses
   Available Model Group Fallbacks=N
   

3. Upstream request to POST https://api.fireworks.ai/inference/v1/responses: input roles ['user', 'developer', 'user']

/v1/chat/completions, developer message after the first user turn, Fireworks

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:57770/v1/chat/completions -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"fireworks-qwen3p8","messages":[{"role":"user","content":"Hi there"},{"role":"developer","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 400
   error: litellm.BadRequestError: Fireworks_aiException - {"error":{"object":"error","type":"invalid_request_error","code":"invalid_request_error","message":"jinja template rendering failed. System message must be at the beginning."}}. Received Model Group=fireworks-qwen3p8
   Available Model Group Fallbacks=No
   

3. Upstream request to POST https://api.fireworks.ai/inference/v1/chat/completions: messages roles ['user', 'system', 'user']

/v1/chat/completions, same messages through openai/ with the Fireworks base URL

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:57770/v1/chat/completions -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"fireworks-qwen3p8-openai-compat","messages":[{"role":"user","content":"Hi there"},{"role":"developer","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 400
   error: litellm.BadRequestError: OpenAIException - jinja template rendering failed. System message must be at the beginning.. Received Model Group=fireworks-qwen3p8-openai-compat
   Available Model Group Fallbacks=None
   

3. Upstream request to POST https://api.fireworks.ai/inference/v1/chat/completions: messages roles ['user', 'system', 'user']

/v1/chat/completions, same messages to OpenAI (control)

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:57770/v1/chat/completions -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"gpt-5.4-mini","messages":[{"role":"user","content":"Hi there"},{"role":"developer","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 200
   assistant: Paris
   

3. Upstream request to POST https://api.openai.com/v1/chat/completions: messages roles ['user', 'system', 'user']

/v1/chat/completions, same messages through custom_openai/

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:57770/v1/chat/completions -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"qwen3p8-custom-openai","messages":[{"role":"user","content":"Hi there"},{"role":"developer","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 400
   error: litellm.BadRequestError: Custom_openaiException - jinja template rendering failed. System message must be at the beginning.. Received Model Group=qwen3p8-custom-openai
   Available Model Group Fallbacks=None
   

3. Upstream request to POST https://api.fireworks.ai/inference/v1/chat/completions: messages roles ['user', 'system', 'user']

/v1/chat/completions, same messages through hosted_vllm/

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:57770/v1/chat/completions -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"qwen3p8-hosted-vllm","messages":[{"role":"user","content":"Hi there"},{"role":"developer","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 400
   error: litellm.BadRequestError: Hosted_vllmException - {"error":{"object":"error","type":"invalid_request_error","code":"invalid_request_error","message":"jinja template rendering failed. System message must be at the beginning."}}. Received Model Group=qwen3p8-hosted-vllm
   Available Model Group Fallbacks=N
   

3. Upstream request to POST https://api.fireworks.ai/inference/v1/chat/completions: messages roles ['user', 'system', 'user']

/v1/chat/completions, the second commenter's two leading system messages, custom_openai/

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:57770/v1/chat/completions -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"qwen3p8-custom-openai","messages":[{"role":"system","content":"You are a terse assistant."},{"role":"system","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 200
   assistant: Paris
   

3. Upstream request to POST https://api.fireworks.ai/inference/v1/chat/completions: messages roles ['system', 'system', 'user'], first system content 'You are a terse assistant.'

/v1/chat/completions, the same two leading system messages, hosted_vllm/

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:57770/v1/chat/completions -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"qwen3p8-hosted-vllm","messages":[{"role":"system","content":"You are a terse assistant."},{"role":"system","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 200
   assistant: Paris
   

3. Upstream request to POST https://api.fireworks.ai/inference/v1/chat/completions: messages roles ['system', 'system', 'user'], first system content 'You are a terse assistant.'

/v1/messages, system prompt with a multi-turn conversation (control, this endpoint has no developer role)

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:57770/v1/messages -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"fireworks-qwen3p8","max_tokens":64,"system":"Answer with exactly one word.","messages":[{"role":"user","content":"Hi there"},{"role":"assistant","content":"Hello!"},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 200
   assistant: [None, 'Paris']
   

3. Upstream request to POST https://api.fireworks.ai/inference/v1/chat/completions: messages roles ['system', 'user', 'assistant', 'user'], first system content 'Answer with exactly one word.'

/v1/responses, instructions plus a leading system item plus a later developer item, native route

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:57770/v1/responses -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"fireworks-qwen3p8","instructions":"You are a terse assistant.","input":[{"role":"system","content":"Keep answers short."},{"role":"user","content":"Hi there"},{"role":"developer","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 200
   output_text: ['Paris']
   

3. Upstream request to POST https://api.fireworks.ai/inference/v1/responses: input roles ['user', 'user'], instructions 'You are a terse assistant.\n\nKeep answers short.\n\nAnswer with exactly one word.'

Codex CLI, chat completions bridge deployment

1. With model = "fireworks-qwen3p8-bridge" and base_url = "http://localhost:57770/v1", run LITELLM_QA_KEY=sk-1234 codex in a repo and type What is the capital of France? Answer with exactly one word.
2. Observed: the pane shows the gateway's 400 in red, jinja template rendering failed. System message must be at the beginning., and no reply
3. Type And of Germany? Answer with exactly one word.
4. Observed: the same 400 again, the session never gets an answer
5. Upstream requests to POST https://api.fireworks.ai/inference/v1/chat/completions: messages roles ['system', 'system', 'user', 'user'] on the first turn (Codex's instructions and its developer item as two separate system messages), then ['system', 'system', 'user', 'user', 'user']

Codex CLI, native Fireworks Responses deployment

1. With model = "fireworks-qwen3p8" and base_url = "http://localhost:57770/v1", run LITELLM_QA_KEY=sk-1234 codex in a repo and type What is the capital of France? Answer with exactly one word.
2. Observed: Paris
3. Type And of Germany? Answer with exactly one word.
4. Observed: Berlin
5. Upstream requests to POST https://api.fireworks.ai/inference/v1/responses: input roles ['user', 'user'] with Codex's instructions and developer item folded into instructions (29714 chars), then ['user', 'user', 'reasoning', 'assistant', 'user']

After (8b49a64)

/v1/responses, developer item after the first input item, native Fireworks Responses route

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:48910/v1/responses -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"fireworks-qwen3p8","input":[{"role":"user","content":"Hi there"},{"role":"developer","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 200
   output_text: ['Paris']
   

3. Upstream request to POST https://api.fireworks.ai/inference/v1/responses: input roles ['user', 'user'], instructions 'Answer with exactly one word.'

/v1/responses, same input through the chat completions bridge

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:48910/v1/responses -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"fireworks-qwen3p8-bridge","input":[{"role":"user","content":"Hi there"},{"role":"developer","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 200
   output_text: ['Paris']
   

3. Upstream request to POST https://api.fireworks.ai/inference/v1/chat/completions: messages roles ['system', 'user', 'user'], first system content 'Answer with exactly one word.'

/v1/responses, same input to OpenAI (control)

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:48910/v1/responses -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"gpt-5.4-mini","input":[{"role":"user","content":"Hi there"},{"role":"developer","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 200
   output_text: ['Paris']
   

3. Upstream request to POST https://api.openai.com/v1/responses: input roles ['user', 'developer', 'user']

/v1/responses, same input through the reporter's custom_openai/ prefix (chat completions bridge)

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:48910/v1/responses -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"qwen3p8-custom-openai","input":[{"role":"user","content":"Hi there"},{"role":"developer","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 200
   output_text: ['Paris']
   

3. Upstream request to POST https://api.fireworks.ai/inference/v1/chat/completions: messages roles ['system', 'user', 'user'], first system content 'Answer with exactly one word.'

/v1/responses, same input through hosted_vllm/ on the chat completions bridge

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:48910/v1/responses -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"qwen3p8-hosted-vllm","input":[{"role":"user","content":"Hi there"},{"role":"developer","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 200
   output_text: ['Paris']
   

3. Upstream request to POST https://api.fireworks.ai/inference/v1/chat/completions: messages roles ['system', 'user', 'user'], first system content 'Answer with exactly one word.'

/v1/responses, same input through hosted_vllm/ on its native Responses route (pre-existing, this PR leaves it alone)

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:48910/v1/responses -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"qwen3p8-hosted-vllm-native-responses","input":[{"role":"user","content":"Hi there"},{"role":"developer","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 400
   error: litellm.BadRequestError: Hosted_vllmException - {"error":{"type":"invalid_request_error","code":"invalid_request_error","message":"jinja template rendering failed. System message must be at the beginning."}}. Received Model Group=qwen3p8-hosted-vllm-native-responses
   Available Model Group Fallbacks=N
   

3. Upstream request to POST https://api.fireworks.ai/inference/v1/responses: input roles ['user', 'developer', 'user']

/v1/chat/completions, developer message after the first user turn, Fireworks

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:48910/v1/chat/completions -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"fireworks-qwen3p8","messages":[{"role":"user","content":"Hi there"},{"role":"developer","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 200
   assistant: Paris
   

3. Upstream request to POST https://api.fireworks.ai/inference/v1/chat/completions: messages roles ['system', 'user', 'user'], first system content 'Answer with exactly one word.'

/v1/chat/completions, same messages through openai/ with the Fireworks base URL

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:48910/v1/chat/completions -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"fireworks-qwen3p8-openai-compat","messages":[{"role":"user","content":"Hi there"},{"role":"developer","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 200
   assistant: Paris
   

3. Upstream request to POST https://api.fireworks.ai/inference/v1/chat/completions: messages roles ['system', 'user', 'user'], first system content 'Answer with exactly one word.'

/v1/chat/completions, same messages to OpenAI (control)

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:48910/v1/chat/completions -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"gpt-5.4-mini","messages":[{"role":"user","content":"Hi there"},{"role":"developer","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 200
   assistant: Paris
   

3. Upstream request to POST https://api.openai.com/v1/chat/completions: messages roles ['user', 'system', 'user']

/v1/chat/completions, same messages through custom_openai/

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:48910/v1/chat/completions -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"qwen3p8-custom-openai","messages":[{"role":"user","content":"Hi there"},{"role":"developer","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 200
   assistant: Paris
   

3. Upstream request to POST https://api.fireworks.ai/inference/v1/chat/completions: messages roles ['system', 'user', 'user'], first system content 'Answer with exactly one word.'

/v1/chat/completions, same messages through hosted_vllm/

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:48910/v1/chat/completions -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"qwen3p8-hosted-vllm","messages":[{"role":"user","content":"Hi there"},{"role":"developer","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 200
   assistant: Paris
   

3. Upstream request to POST https://api.fireworks.ai/inference/v1/chat/completions: messages roles ['system', 'user', 'user'], first system content 'Answer with exactly one word.'

/v1/chat/completions, the second commenter's two leading system messages, custom_openai/

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:48910/v1/chat/completions -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"qwen3p8-custom-openai","messages":[{"role":"system","content":"You are a terse assistant."},{"role":"system","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 200
   assistant: Paris
   

3. Upstream request to POST https://api.fireworks.ai/inference/v1/chat/completions: messages roles ['system', 'user'], first system content 'You are a terse assistant.\n\nAnswer with exactly one word.'

/v1/chat/completions, the same two leading system messages, hosted_vllm/

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:48910/v1/chat/completions -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"qwen3p8-hosted-vllm","messages":[{"role":"system","content":"You are a terse assistant."},{"role":"system","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 200
   assistant: Paris
   

3. Upstream request to POST https://api.fireworks.ai/inference/v1/chat/completions: messages roles ['system', 'user'], first system content 'You are a terse assistant.\n\nAnswer with exactly one word.'

/v1/messages, system prompt with a multi-turn conversation (control, this endpoint has no developer role)

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:48910/v1/messages -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"fireworks-qwen3p8","max_tokens":64,"system":"Answer with exactly one word.","messages":[{"role":"user","content":"Hi there"},{"role":"assistant","content":"Hello!"},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 200
   assistant: [None, 'Paris']
   

3. Upstream request to POST https://api.fireworks.ai/inference/v1/chat/completions: messages roles ['system', 'user', 'assistant', 'user'], first system content 'Answer with exactly one word.'

/v1/responses, instructions plus a leading system item plus a later developer item, native route

1. Run

   curl -sS -w ' HTTP %{http_code}\n' http://localhost:48910/v1/responses -H 'Authorization: Bearer sk-1234' -H 'content-type: application/json' -d '{"model":"fireworks-qwen3p8","instructions":"You are a terse assistant.","input":[{"role":"system","content":"Keep answers short."},{"role":"user","content":"Hi there"},{"role":"developer","content":"Answer with exactly one word."},{"role":"user","content":"What is the capital of France?"}]}'
   

2. Observed

   HTTP 200
   output_text: ['Paris']
   

3. Upstream request to POST https://api.fireworks.ai/inference/v1/responses: input roles ['user', 'user'], instructions 'You are a terse assistant.\n\nKeep answers short.\n\nAnswer with exactly one word.'

Codex CLI, chat completions bridge deployment

1. With model = "fireworks-qwen3p8-bridge" and base_url = "http://localhost:48910/v1", run LITELLM_QA_KEY=sk-1234 codex in a repo and type What is the capital of France? Answer with exactly one word.
2. Observed: Paris
3. Type And of Germany? Answer with exactly one word.
4. Observed: Berlin
5. Upstream requests to POST https://api.fireworks.ai/inference/v1/chat/completions: messages roles ['system', 'user', 'user'] on the first turn, the one system message carrying Codex's instructions and developer item as a four-block text list, then ['system', 'user', 'user', 'assistant', 'user']

Codex CLI, native Fireworks Responses deployment

1. With model = "fireworks-qwen3p8" and base_url = "http://localhost:48910/v1", run LITELLM_QA_KEY=sk-1234 codex in a repo and type What is the capital of France? Answer with exactly one word.
2. Observed: Paris
3. Type And of Germany? Answer with exactly one word.
4. Observed: Berlin
5. Upstream requests to POST https://api.fireworks.ai/inference/v1/responses: input roles ['user', 'user'] with instructions (29714 chars), then ['user', 'user', 'reasoning', 'assistant', 'user'], byte-for-byte the shapes of the merge base

Observations from the run

• hosted_vllm/ native Responses still 400s on both legs, left alone
• Codex bridge folds developer into a four-block system list
• Codex native second turn works on both legs now
• Fireworks accepted list-content system messages on every route
• openai/ with a Fireworks base URL hoists too, as intended
• /v1/messages on qwen3.8 returns a text-less reasoning block, both legs
• deepseek and mistral flatten folded list content to a string

Notes on the branch history

• origin/litellm_internal_staging was merged in three times, at 1d26ee5 (after the PR went CONFLICTING on tests/test_litellm/llms/openai/test_openai.py, combined with staging's stream-options tests first), at d5d70da (staging head 1009976, which brought the native Fireworks Responses route from feat(fireworks_ai): add native Responses API config #39826), and at 8b49a64 (staging head db7ca65, now the merge base, which brought fix(fireworks_ai): fold instructions and developer items into one leading system message on the Responses path #40268). Merged rather than rebased so the bot reviews and the per-commit QA stay attached to their commits
• 22da54b carried this branch's own hoist onto the native Fireworks Responses route. fix(fireworks_ai): fold instructions and developer items into one leading system message on the Responses path #40268 landed a fold into instructions there first, so the 8b49a64 merge takes staging's transformation and keeps only the shared helper, now hoisted_instruction_indices in base_utils, which both paths import
• 69fd884 is a cherry-pick of test(base_utils): cover the empty-string branch of _text_blocks #39877 by the author of fix(transformation): merge consecutive system and developer messages for non-OpenAI providers and Responses API #39282 (authorship kept): a regression test for the empty-string branch of the text-block fold, the one line Codecov flagged as uncovered. test(base_utils): cover the empty-string branch of _text_blocks #39877 was closed with a comment naming the cherry-picked sha
• Caveats round 2 at 69fd884 raised two Mediums and seven Lows; the fixes are the commit after it: the litellm_proxy/ passthrough override is gone, a null or missing system content inside a run folds away instead of raising, and a message-level cache_control on a list-content member lands on its last block
• ea840eb was pushed with a lint miss (the TYPE_CHECKING import of AllMessageValues that two string annotations still name went out with the removed override); the commit right after it restores the import, nothing else changes

Dependency walk at 8b49a64 (/live-pr-risk)

Verdict: no breaking or backward-incompatible dependent found. The one contract change, the translate_developer_role_to_system_role signature, has five implementers and one caller, all in the diff

• Changed surface: BaseConfig.translate_developer_role_to_system_role gains keyword-only custom_llm_provider and api_base and returns a Sequence. OpenAIGPTConfig, OpenAIConfig, OpenAIOSeriesConfig, and AzureOpenAIConfig override it, and grep -rn "def translate_developer_role_to_system_role" finds exactly those five, with no duck-typed implementer elsewhere. The single caller is completion() in litellm/main.py, which passes both keywords and wraps the result in a list for the legacy handlers. _targets_openai_hosted_endpoint now delegates to is_openai_backed_api_base, so its two other consumers (_should_preserve_cache_control_for_endpoint, _flattened_tools_update_for_openai) move with it. _hoisted_indices in the Fireworks Responses transformation is replaced by the shared hoisted_instruction_indices in base_utils. No config key, env var, Pydantic field, DB column, dashboard field, or string-dispatched name changes
• Dependents by reachability: (1) every proxy route that ends in completion() (/v1/chat/completions, the Responses bridge, /v1/messages) for the 70-odd configs inheriting OpenAIGPTConfig or OpenAIConfig, verified live on the legs above for fireworks_ai, openai, custom_openai, and hosted_vllm and in the SDK sweep below; (2) the native Fireworks Responses route through hoisted_instruction_indices, verified live on both legs; (3) cache_control preservation and tool flattening for openai/ with a custom base, unit-tested in test_openai_gpt_transformation.py, with the three base URL shapes that flip listed in Caveats and not driven since none serves the OpenAI API; (4) tests/llm_translation/base_llm_unit_tests.py::test_developer_role_translation, whose both variants send a leading developer message, where in-place mapping and hoisting agree
• SDK sweep (litellm.completion, base is the main checkout at 2611f64, head is 8b49a64, same venv, litellm.__file__ printed per leg): [user, developer with list content, user] and [user, assistant, developer] against deepseek-v4-flash, mistral-small-latest, openrouter llama-3.1-8b, perplexity sonar, xai grok-4.20, databricks qwen3-next-80b, and fireworks qwen3p8. Every later-developer case answers Paris on both legs; the head sends it as a leading system message ([system, user, user]), the base as [user, system, user]. deepseek and mistral flatten the folded list content to a string in their own transforms, the rest forward the block list and the backend accepts it. The closing-developer case stays in place on both legs and answers Berlin everywhere except fireworks, which 400s on both (the closing-run rule in Caveats)
• Merge-ref check: staging moved from db7ca65 to c005431 after the last merge. A git merge --no-commit of that head into 8b49a64 is conflict-free (only constants.py overlaps, on different lines), and the sweep on that merged tree matches the head leg for fireworks, mistral, xai, and openrouter
• Not verified: nvidia_nim (both llama models answer 410 Gone from the endpoint on base and head alike), a real vLLM backend, Groq and the other string-only system backends (no key), and the three hosted-endpoint URL shapes above

Type

🐛 Bug Fix

Caveats (if any)

Low

• Hoist-and-merge is a no-opt-out behaviour change on every OpenAI-compatible config

  • Every config built on OpenAIGPTConfig or OpenAIConfig hoists, whether or not its template needs it: custom_openai, hosted_vllm, openrouter, xai, mistral, groq, github_copilot, databricks (Databricks-hosted Claude included), azure_ai, litellm_proxy, fireworks_ai, the Vertex model-garden configs (vertex_ai/meta/...), and Bedrock's bedrock/openai/... route among them. A developer message after the first user turn moves to the top and merges into one leading system message (unless it closes the conversation right after an assistant turn), so a mid-session developer instruction, a Codex approval-mode change or an agent framework re-injecting policy, reads as if it had been there since the first turn, with no flag to keep the old in-place mapping. The affected set is exactly the set the merge base already rewrote from developer to system in place, only the position changes; the hosted OpenAI passthrough and the Azure in-place mapping are untouched; the per-message opt-out is to send system, which the hoist never moves
  • Left as is: on a backend that accepts system anywhere the request still succeeds and the model still receives the instruction, so the only observable difference is its position, and each of the three ways to remove it costs more than it returns. A litellm_settings switch is a global knob nobody asked for that every OpenAI-compatible user would have to learn about to get the fix. A curated list of template-strict providers goes stale the day a new backend appears and misses openrouter, custom_openai, and hosted_vllm, whose upstream is whatever the user points them at. A send-then-retry on the jinja 400 doubles the call on the failing path and keys on one backend's error string

• openai/ behind a non-openai.com host hoists too

  • The hosted-endpoint check is False for Helicone, Cloudflare AI Gateway, or a corporate proxy base URL, so openai/gpt-4.1 through those hoists (['system','user','assistant','user']) even though the upstream is the real OpenAI API that accepts developer anywhere; OpenAI tolerates a trailing assistant, so it is a semantic shift there rather than a 400
  • Left as is: a gateway URL says nothing about which upstream sits behind it, so the only way to keep the passthrough there is a user-declared "this base is really OpenAI" setting, the same unrequested knob as above. The hoisted request is one OpenAI accepts, and the per-message opt-out (send system) applies here too

• A real vLLM backend was not driven

  • The reporter's stack is custom_openai/ and the second commenter's is vLLM. Both configs were driven live against Fireworks' Qwen3.8, whose jinja template rejects a later system message the same way (400 on the merge base, 200 on the tip, legs above); vLLM's own rendering of that template is unverified. Codex was driven with its developer-first shape, and the ticket's mid-session shape is the ticket's own curl

• Client-authored mid-conversation system messages still fail on strict templates

  • system, user, system on Fireworks Qwen3.8 is 400 "System message must be at the beginning" on both legs. The hoist is developer-only by design, so clients that already translate to system themselves are not helped

• A closing developer right after an assistant turn still 400s on Fireworks

  • The closing-run rule is deliberate and unit-tested; the live legs at 4e3ea74 showed the jinja 400 on both sides for that shape

• The hosted-endpoint check changed for three base URL shapes

  • _targets_openai_hosted_endpoint now delegates to is_openai_backed_api_base, so a bare openai.com host, an openai.com subdomain outside api.openai.com, and a scheme-less base count as compatible instead of hosted, which also flips cache_control preservation and tool flattening for them. None of the three serves the OpenAI API (a scheme-less base does not work with httpx at all), and a second hostname rule would bring back the duplicate the consolidation removed

• Only the native Fireworks Responses route folds developer items

  • The other native Responses configs (hosted_vllm, xai, openrouter, litellm_proxy, azure, chatgpt, volcengine, perplexity, manus, github_copilot, openai_like) forward developer items as they arrive; hosted_vllm/ on its native route against Fireworks' native Responses API still 400s on both legs (case above). A backend that implements the Responses API owns its own templating, so the bridge is the route for a system-first template there, and fix(fireworks_ai): fold instructions and developer items into one leading system message on the Responses path #40268's fold stays Fireworks-only

• Merged system content becomes a block list whenever any member has list content, cache_control, or billing metadata

  • Codex always sends developer items as input_text lists, so its first turn yields content: [{type: text, ...}, ...] even when every block is plain text (the four-block system message in the Codex bridge case above). All six live providers accepted array system content, so string-only backends (Groq class, no key available) are unverified rather than reproduced. A message-level cache_control is dropped when the run's last block is not a dict or already carries one

• Folding keeps only the last name of a system run

  • The merged message keeps the name of the last named member only

• Consecutive client-authored system messages fold anywhere in the conversation, not only at the top

  • [system, user, system, system, user] becomes [system, user, system, user] with the two middle messages joined by a blank line; they stay in place, so a strict template still rejects them

• Three non-required CircleCI jobs and osv-scan are red at the tip, all red on staging too

  • llm_translation_testing (three mocked completion tests) and local_testing_part1 (test_completion_novita_ai) fail identically on staging's latest pipeline since fix(convert_dict_to_response): handle empty choices list without raising 500 APIError #40294 started raising on a response with no choices (tracked internally); local_testing_part2's test_openai_stream_options_call_text_completion was a mid-stream connection error on a text completion path this PR does not touch and is green on staging; osv-scan flags smol-toml in the dashboard lockfile, which this PR does not change, and is red on every recent staging head. Every required check passes

• No docs change for the new developer-role semantics

  • Docs live in BerriAI/litellm-docs, so the diff carries no docs; the docs follow-up is LIT-7069

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

• 8b49a64 passes /live-pr-risk

---

Note

Medium Risk
Changes message ordering and merging for many chat completion paths; semantics shift for mid-conversation developer instructions on compatible backends, though behavior is extensively tested.

Overview
Fixes 400 "System message must be at the beginning" on OpenAI-compatible backends when a developer message (or multiple leading system messages) appears after the first user turn.

Provider-aware developer handling: translate_developer_role_to_system_role now receives custom_llm_provider and api_base. Hosted OpenAI and Azure OpenAI still only remap developer → system in place; o-series leaves developer unchanged. OpenAI-compatible routes (Fireworks, vLLM, custom_openai, Azure AI, litellm_proxy, etc.) call new logic in base_utils that hoists mid-conversation developer messages into the leading instruction block, merges consecutive system/developer runs into one leading system message, and leaves in place a developer block that closes the thread right after an assistant turn. completion() passes the new kwargs through from main.py.

Shared folding rules: hoisted_instruction_indices is centralized in base_utils and reused by the Fireworks Responses path (replacing a local duplicate). Merging preserves cache_control on text blocks and keeps Anthropic billing header lines as separate blocks via ANTHROPIC_BILLING_METADATA_PREFIX.

Tests cover hoist/merge edge cases, Azure vs Azure AI vs OpenAI hosted vs compatible api_base, proxy passthrough, and Responses instructions + developer input.

^{Reviewed by Cursor Bugbot for commit 8b49a64. Bugbot is set up for automated code reviews on this repo. Configure here.}

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_lit_7019_hoist_developer_messages (8b49a64) with litellm_internal_staging (2611f64)}

[greptile-apps]

Greptile Summary

This PR makes developer-role translation provider-aware and folds applicable developer instructions into one leading system message for OpenAI-compatible chat backends

• Preserves existing behavior for hosted OpenAI, Azure OpenAI, and OpenAI o-series models
• Centralizes instruction selection and reuses it in the native Fireworks Responses transformation
• Preserves cache-control and Anthropic billing metadata while merging message content
• Adds regression coverage for ordering, merging, metadata, provider routing, and Responses bridging

Confidence Score: 5/5

The PR appears safe to merge, with no outstanding previous findings or accepted new issues

The current head is unchanged since the previous review. The earlier billing-metadata and source-comment findings are fixed, the Ollama and trailing-instruction findings were withdrawn after provider-specific validation, and the Fireworks Responses thread is resolved with the shared folding behavior present

Important Files Changed

Filename	Overview
litellm/llms/base_llm/base_utils.py	Adds shared developer-message hoisting, system-run merging, closing-item selection, and metadata-preserving content folding
litellm/llms/openai/chat/gpt_transformation.py	Selects in-place mapping for hosted OpenAI and hoisting for OpenAI-compatible endpoints
litellm/llms/azure/chat/gpt_transformation.py	Keeps Azure OpenAI developer messages in their original positions while translating their role
litellm/llms/fireworks_ai/responses/transformation.py	Replaces the route-local instruction-index logic with the shared helper
litellm/main.py	Passes provider and API-base context into provider-specific developer-role translation
tests/test_litellm/llms/base_llm/test_base_utils.py	Covers message ordering, system folding, closing developer runs, cache control, null content, names, and billing metadata

_{Reviews (11): Last reviewed commit: "Merge branch 'litellm_internal_staging' ..." | Re-trigger Greptile}

[mateo-berri]

@greptileai

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

[mateo-berri]

@greptileai

[codecov]

Codecov Report

❌ Patch coverage is 97.54098% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/llms/base_llm/base_utils.py	97.75%	2 Missing ⚠️
...perimental_pass_through/messages/transformation.py	66.66%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 8b49a64. Configure here.}

[tin-berri]

LGTM