Skip to content

fix(responses-bridge): extract list-format system content into instructions#21192

Merged
krrishdholakia merged 24 commits intoBerriAI:litellm_oss_staging_02_14_2026from
jo-nike:fix/responses-bridge-list-system-content
Feb 14, 2026
Merged

fix(responses-bridge): extract list-format system content into instructions#21192
krrishdholakia merged 24 commits intoBerriAI:litellm_oss_staging_02_14_2026from
jo-nike:fix/responses-bridge-list-system-content

Conversation

@jo-nike
Copy link

@jo-nike jo-nike commented Feb 14, 2026

Summary

When system message content is a list of content blocks (e.g. [{"type": "text", "text": "..."}]) instead of a plain string, the responses API bridge in convert_chat_completion_messages_to_responses_api() was passing it through as a role: system message in the input items. APIs like ChatGPT Codex reject this with "System messages are not allowed".

This happens when requests arrive via the Anthropic /v1/messages adapter, which converts system prompts into list-format content blocks in the OpenAI chat completions format before the responses bridge processes them.

Fix

Extract text from list content blocks and concatenate into the instructions parameter, matching the existing behavior for string system content. The else branch that added list system content as input items is replaced with an elif isinstance(content, list) branch that joins text parts.

How to reproduce

  1. Configure a ChatGPT Codex model (e.g. chatgpt/gpt-5.2-codex) with mode: "responses"
  2. Send a request via the Anthropic /v1/messages endpoint with a system message
  3. The Anthropic adapter converts it to chat completions format with list content blocks
  4. The responses bridge passes the list content as role: system in input items
  5. ChatGPT Codex rejects with "System messages are not allowed"

Changes

  • litellm/completion_extras/litellm_responses_transformation/transformation.py: Replace the else branch (which appended list system content as input items) with an elif isinstance(content, list) branch that extracts text from content blocks into instructions

yuneng-jiang and others added 22 commits February 13, 2026 16:20
…h_route (BerriAI#20849)

* fix: add custom_body parameter to endpoint_func in create_pass_through_route

The bedrock_proxy_route calls `endpoint_func(custom_body=data)` to
pass a pre-parsed, SigV4-signed request body. However, the
`endpoint_func` closure created by `create_pass_through_route` does
not accept a `custom_body` keyword argument, causing:

    TypeError: endpoint_func() got an unexpected keyword argument 'custom_body'

Add `custom_body: Optional[dict] = None` to both `endpoint_func`
definitions (adapter-based and URL-based). In the URL-based path,
when `custom_body` is provided by the caller, use it instead of
re-parsing the body from the raw request.

Fixes BerriAI#16999

* Add tests for custom_body handling in create_pass_through_route

Address reviewer feedback on PR BerriAI#20849:

- Document why the adapter-based endpoint_func accepts custom_body
  for signature compatibility but does not forward it (the underlying
  chat_completion_pass_through_endpoint does not support it).
- Add test_create_pass_through_route_custom_body_url_target: verifies
  that when a caller (e.g. bedrock_proxy_route) supplies custom_body,
  it takes precedence over the body parsed from the raw request.
- Add test_create_pass_through_route_no_custom_body_falls_back:
  verifies that the default path (no custom_body) correctly uses the
  request-parsed body, preserving existing behavior.

Both tests are fully mocked following the project's CONTRIBUTING.md
guidelines and the patterns established in the existing test file.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: themavik <themavik@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
…erriAI#21169)

* fix: populate identity fields in proxy admin JWT early-return path

When is_proxy_admin is True, the UserAPIKeyAuth early-return now includes
user_id, team_id, team_alias, team_metadata, org_id, and end_user_id
resolved from the JWT. Previously only user_role and parent_otel_span
were set, causing blank Team Name and Internal User in Request Logs UI.

* test: add unit tests for proxy admin JWT identity fields
[Feature] UI - Access Groups: Table and Details Page
…name

[Refactor] Access Group model_ids to model_names for backwards Compatability
* Pyroscope: require PYROSCOPE_APP_NAME and PYROSCOPE_SERVER_ADDRESS, add UTF-8 locale hint

- No defaults for PYROSCOPE_APP_NAME or PYROSCOPE_SERVER_ADDRESS; fail at startup if unset when Pyroscope is enabled
- Set LANG/LC_ALL to C.UTF-8 when unset to reduce malformed_profile (invalid UTF-8) rejections
- Startup message suggests PYTHONUTF8=1 if server rejects profiles
- Simplify LITELLM_ENABLE_PYROSCOPE in config_settings; document Pyroscope env vars as required with no default
- Add pyroscope_profiling to sidebar (Alerting & Monitoring)
- pyproject.toml: pyroscope-io as required dep on non-Windows (marker), in proxy extra

* proxy: add PYROSCOPE_SAMPLE_RATE env, use verbose logging, fix int type

- Add optional PYROSCOPE_SAMPLE_RATE env (integer, no default)
- Pass sample_rate to pyroscope.configure() as int for pyroscope-io
- Replace print with verbose_proxy_logger (info/warning)
- Document PYROSCOPE_SAMPLE_RATE in config_settings.md

* Address Greptile PR feedback: Pyroscope optional, docs, tests, docstring

- pyproject.toml: mark pyroscope-io as optional=true (proxy extra only)
- Add docs/my-website/docs/proxy/pyroscope_profiling.md (fix broken sidebar link)
- Add tests/test_litellm/proxy/test_pyroscope.py for _init_pyroscope()
- proxy_server: fix _init_pyroscope docstring (required server/app name, sample rate as int)

* Update litellm/proxy/proxy_server.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Several Gemini models (TTS, native-audio, robotics, gemma) were missing
tpm/rpm values, causing test_get_model_info_gemini to fail.

Added conservative default values (tpm=250000, rpm=10) for preview models.
gemini-2.5-flash-preview-tts gets tpm=4000000, rpm=10.

Co-authored-by: OpenClaw <openclaw@users.noreply.github.com>
…erriAI#21178)

Co-authored-by: shin-bot-litellm <shin-bot-litellm@users.noreply.github.com>
- vertex_ai/gemini: fix TypedDict assignment via explicit dict cast
- mcp_server: convert MutableMapping scope to dict for type safety
- pass_through_endpoints: simplify custom_body logic to fix type narrowing
- vector_store_endpoints: add Any annotation for dynamic hook return
- responses transformation: use dict() for Reasoning and setattr for dynamic field
- zscaler_ai_guard: add assert for api_base None check

Co-authored-by: shin-bot-litellm <shin-bot-litellm@users.noreply.github.com>
* fix(ci): Fix ruff lint error - unused import

Remove unused 'cast' import in vertex_ai_ingestion.py (ruff F401)

* fix(ci): Fix E2E login button selector - use exact match

Login button selector now matches both 'Login' and 'Login with SSO',
causing strict mode violation. Use { exact: true } to match only 'Login'.

---------

Co-authored-by: OpenClaw <openclaw@users.noreply.github.com>
- vertex_ai/gemini/transformation.py: Fix TypedDict assignment via dict alias
- mcp_server/server.py: Convert ASGI scope to dict for type compatibility
- pass_through_endpoints.py: Add explicit Optional[dict] type annotation
- vector_store_endpoints/endpoints.py: Add Any type for dynamic proxy hook
- responses transformation.py: Use dict(Reasoning()) and setattr for compatibility
- zscaler_ai_guard.py: Add assert for api_base nullability

Co-authored-by: OpenClaw <openclaw@users.noreply.github.com>
…l execution (BerriAI#21177)

* Add pipeline type definitions for guardrail pipelines

PipelineStep, GuardrailPipeline, PipelineStepResult, PipelineExecutionResult
with validation for actions (allow/block/next/modify_response) and modes.

* Export pipeline types from policy_engine types package

* Add optional pipeline field to Policy model

* Add pipeline executor for sequential guardrail execution

* Parse pipeline config in policy registry

* Add pipeline validation in policy validator

* Add pipeline resolution and managed guardrail tracking

* Resolve pipelines and exclude managed guardrails in pre-call

* Integrate pipeline execution into proxy pre_call_hook

* Add test guardrails for pipeline E2E testing

* Add example pipeline config YAML

* Add unit tests for pipeline type definitions

* Add unit tests for pipeline executor

* Update litellm/proxy/policy_engine/pipeline_executor.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update litellm/proxy/utils.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Add pipeline type definitions for guardrail pipelines

PipelineStep, GuardrailPipeline, PipelineStepResult, PipelineExecutionResult
with validation for actions (allow/block/next/modify_response) and modes.

* Export pipeline types from policy_engine types package

* Add optional pipeline field to Policy model

* Add pipeline executor for sequential guardrail execution

* Parse pipeline config in policy registry

* Add pipeline validation in policy validator

* Add pipeline resolution and managed guardrail tracking

* Resolve pipelines and exclude managed guardrails in pre-call

* Integrate pipeline execution into proxy pre_call_hook

* Add test guardrails for pipeline E2E testing

* Add example pipeline config YAML

* Add unit tests for pipeline type definitions

* Add unit tests for pipeline executor

* Add pipeline column to LiteLLM_PolicyTable schema

* Add pipeline field to policy CRUD request/response types

* Add pipeline support to policy DB CRUD operations

* Add PipelineStep and GuardrailPipeline TypeScript types

* Add Zapier-style pipeline flow builder UI component

* Integrate pipeline flow builder with mode toggle in policy form

* Add pipeline display section to policy info view

* Add unit tests for pipeline in policy CRUD types

* Refactor policy form to show mode picker first with icon cards

* Add full-screen FlowBuilderPage component for pipeline editing

* Wire up full-screen flow builder in PoliciesPanel with edit routing

* Restyle flow builder to match dev-tool UI aesthetic

* Restyle flow builder cards to match reference design

* Update step card to expanded layout with stacked ON PASS / ON FAIL sections

* Add end card to flow builder showing return to normal control flow

* Add PipelineTestRequest type for test-pipeline endpoint

* Export PipelineTestRequest from policy_engine types

* Add POST /policies/test-pipeline endpoint

* Add testPipelineCall networking function

* Add PipelineStepResult and PipelineTestResult types

* Add test pipeline panel to flow builder with run button and results display

* Fix pipeline executor: inject guardrail name into metadata so should_run_guardrail allows execution

* Update litellm/proxy/policy_engine/pipeline_executor.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update litellm/proxy/utils.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update litellm/proxy/policy_engine/policy_endpoints.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update litellm/proxy/policy_engine/pipeline_executor.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
…ctions

When system message content is a list of content blocks
(e.g. [{"type": "text", "text": "..."}]) instead of a plain string,
the responses API bridge was passing it through as a role: system
message in the input items. APIs like ChatGPT Codex reject this
with "System messages are not allowed".

This happens when requests come through the Anthropic /v1/messages
adapter, which converts system prompts into list-format content blocks
in the OpenAI chat completions format.

Fix: extract text from list content blocks and concatenate into the
instructions parameter, matching the existing behavior for string
system content.
@vercel
Copy link

vercel bot commented Feb 14, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Error Error Feb 14, 2026 6:47am

Request Review

@CLAassistant
Copy link

CLAassistant commented Feb 14, 2026

CLA assistant check
All committers have signed the CLA.

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Feb 14, 2026

Greptile Overview

Greptile Summary

Fixes a bug where system messages with list-format content blocks (e.g. [{"type": "text", "text": "..."}]) were incorrectly passed through as role: system input items in the responses API bridge, causing "System messages are not allowed" errors on APIs like ChatGPT Codex. The fix extracts text from these content blocks and concatenates them into the instructions parameter, matching the existing behavior for string system content.

  • Correctly handles the Anthropic /v1/messages adapter flow, which converts system prompts into list-format content blocks before the responses bridge processes them
  • The extraction logic handles both {"type": "text", "text": "..."} dicts and plain strings within the list
  • Missing tests: No unit tests are included for the new list-content system message handling path, and the existing test suite has no coverage for system message extraction at all
  • Minor: The change removes the else fallback that previously handled unexpected content types, which are now silently ignored

Confidence Score: 4/5

  • This PR is safe to merge — the logic change is correct and well-scoped, though it would benefit from test coverage before merging.
  • The fix is a straightforward and correct change to a single method. The new elif isinstance(content, list) branch properly extracts text from content blocks into the instructions parameter, matching the existing string handling. The only concern is the lack of unit tests, which is important for a regression fix. The change is minimal and unlikely to cause side effects.
  • No files require special attention beyond the missing test coverage for litellm/completion_extras/litellm_responses_transformation/transformation.py.

Important Files Changed

Filename Overview
litellm/completion_extras/litellm_responses_transformation/transformation.py Replaces the catch-all else branch for system messages with an explicit elif isinstance(content, list) that extracts text from content blocks into instructions. The logic is correct but lacks test coverage and silently drops unrecognized content types.

Flowchart

flowchart TD
    A[Anthropic /v1/messages request<br/>with system prompt] --> B[Anthropic Adapter<br/>_add_system_message_to_messages]
    B -->|string system| C[ChatCompletionSystemMessage<br/>content = string]
    B -->|list system blocks| D["ChatCompletionSystemMessage<br/>content = [{type: text, text: ...}]"]
    C --> E[Responses Bridge<br/>convert_chat_completion_messages_to_responses_api]
    D --> E
    E -->|isinstance str| F[instructions = content]
    E -->|isinstance list NEW| G["Extract text from blocks<br/>join into instructions"]
    E -->|else REMOVED| H["❌ Old: passed as role:system input item<br/>→ 'System messages not allowed' error"]
    F --> I[Responses API Request<br/>instructions parameter set]
    G --> I
    I --> J[API call succeeds]
Loading

Last reviewed commit: 19ef616

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 file reviewed, 2 comments

Edit Code Review Agent Settings | Greptile

Comment on lines +166 to +178
elif isinstance(content, list):
# Extract text from content blocks (e.g. [{"type": "text", "text": "..."}])
text_parts = []
for block in content:
if isinstance(block, dict) and block.get("type") == "text":
text_parts.append(block.get("text", ""))
elif isinstance(block, str):
text_parts.append(block)
extracted = " ".join(text_parts)
if instructions:
instructions = f"{instructions} {extracted}"
else:
instructions = extracted
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing test coverage for new branch

This PR adds handling for list-format system content, but there are no corresponding unit tests. The existing test file (tests/test_litellm/completion_extras/litellm_responses_transformation/test_completion_extras_litellm_responses_transformation_transformation.py) has no tests for system message handling at all — neither for string content nor for this new list-content path.

Please add a test case that verifies:

  1. A system message with content: [{"type": "text", "text": "Hello"}, {"type": "text", "text": "World"}] produces instructions = "Hello World" and no system input items.
  2. A system message with list content combined with another string system message properly concatenates into instructions.

This would help prevent regressions and satisfy the PR template requirement of "Add at least 1 test in tests/litellm/".

Context Used: Rule from dashboard - What: Ensure that any PR claiming to fix an issue includes evidence that the issue is resolved, such... (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Comment on lines +166 to +178
elif isinstance(content, list):
# Extract text from content blocks (e.g. [{"type": "text", "text": "..."}])
text_parts = []
for block in content:
if isinstance(block, dict) and block.get("type") == "text":
text_parts.append(block.get("text", ""))
elif isinstance(block, str):
text_parts.append(block)
extracted = " ".join(text_parts)
if instructions:
instructions = f"{instructions} {extracted}"
else:
instructions = extracted
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Silently drops non-str/list system content

The old else branch handled any non-string content type by passing it through as an input item. The new code only handles str and list — any other content type (e.g., a dict or unexpected type) is now silently ignored. While this is unlikely to occur given the AllMessageValues typing, consider adding a final else clause with a warning log for defensive robustness:

Suggested change
elif isinstance(content, list):
# Extract text from content blocks (e.g. [{"type": "text", "text": "..."}])
text_parts = []
for block in content:
if isinstance(block, dict) and block.get("type") == "text":
text_parts.append(block.get("text", ""))
elif isinstance(block, str):
text_parts.append(block)
extracted = " ".join(text_parts)
if instructions:
instructions = f"{instructions} {extracted}"
else:
instructions = extracted
elif isinstance(content, list):
# Extract text from content blocks (e.g. [{"type": "text", "text": "..."}])
text_parts = []
for block in content:
if isinstance(block, dict) and block.get("type") == "text":
text_parts.append(block.get("text", ""))
elif isinstance(block, str):
text_parts.append(block)
extracted = " ".join(text_parts)
if instructions:
instructions = f"{instructions} {extracted}"
else:
instructions = extracted
else:
verbose_logger.warning(
f"Unexpected system message content type: {type(content)}"
)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@jo-nike
Copy link
Author

jo-nike commented Feb 14, 2026

Note: All CI failures are pre-existing on main and unrelated to this change:

  • lint: proxy/litellm_pre_call_utils.py (too many statements) and proxy/policy_engine/pipeline_executor.py (unused variable) — files not touched by this PR
  • test (all): poetry.lock out of sync with pyproject.toml

The unit-test and validate-model-prices-json checks both pass.

Add three tests for convert_chat_completion_messages_to_responses_api:
- String system content → instructions
- List-format content blocks → instructions (the bug this PR fixes)
- Multiple system messages (mixed string and list) concatenated
Address review feedback: add an else clause that logs a warning
for any system content that is neither str nor list, rather than
silently dropping it.
@krrishdholakia krrishdholakia changed the base branch from main to litellm_oss_staging_02_14_2026 February 14, 2026 07:08
@krrishdholakia krrishdholakia merged commit 495ce34 into BerriAI:litellm_oss_staging_02_14_2026 Feb 14, 2026
4 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants