fix(responses-bridge): extract list-format system content into instructions#21192
Conversation
…h_route (BerriAI#20849) * fix: add custom_body parameter to endpoint_func in create_pass_through_route The bedrock_proxy_route calls `endpoint_func(custom_body=data)` to pass a pre-parsed, SigV4-signed request body. However, the `endpoint_func` closure created by `create_pass_through_route` does not accept a `custom_body` keyword argument, causing: TypeError: endpoint_func() got an unexpected keyword argument 'custom_body' Add `custom_body: Optional[dict] = None` to both `endpoint_func` definitions (adapter-based and URL-based). In the URL-based path, when `custom_body` is provided by the caller, use it instead of re-parsing the body from the raw request. Fixes BerriAI#16999 * Add tests for custom_body handling in create_pass_through_route Address reviewer feedback on PR BerriAI#20849: - Document why the adapter-based endpoint_func accepts custom_body for signature compatibility but does not forward it (the underlying chat_completion_pass_through_endpoint does not support it). - Add test_create_pass_through_route_custom_body_url_target: verifies that when a caller (e.g. bedrock_proxy_route) supplies custom_body, it takes precedence over the body parsed from the raw request. - Add test_create_pass_through_route_no_custom_body_falls_back: verifies that the default path (no custom_body) correctly uses the request-parsed body, preserving existing behavior. Both tests are fully mocked following the project's CONTRIBUTING.md guidelines and the patterns established in the existing test file. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: themavik <themavik@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com>
…erriAI#21169) * fix: populate identity fields in proxy admin JWT early-return path When is_proxy_admin is True, the UserAPIKeyAuth early-return now includes user_id, team_id, team_alias, team_metadata, org_id, and end_user_id resolved from the JWT. Previously only user_role and parent_otel_span were set, causing blank Team Name and Internal User in Request Logs UI. * test: add unit tests for proxy admin JWT identity fields
[Feature] UI - Access Groups: Table and Details Page
…name [Refactor] Access Group model_ids to model_names for backwards Compatability
* Pyroscope: require PYROSCOPE_APP_NAME and PYROSCOPE_SERVER_ADDRESS, add UTF-8 locale hint - No defaults for PYROSCOPE_APP_NAME or PYROSCOPE_SERVER_ADDRESS; fail at startup if unset when Pyroscope is enabled - Set LANG/LC_ALL to C.UTF-8 when unset to reduce malformed_profile (invalid UTF-8) rejections - Startup message suggests PYTHONUTF8=1 if server rejects profiles - Simplify LITELLM_ENABLE_PYROSCOPE in config_settings; document Pyroscope env vars as required with no default - Add pyroscope_profiling to sidebar (Alerting & Monitoring) - pyproject.toml: pyroscope-io as required dep on non-Windows (marker), in proxy extra * proxy: add PYROSCOPE_SAMPLE_RATE env, use verbose logging, fix int type - Add optional PYROSCOPE_SAMPLE_RATE env (integer, no default) - Pass sample_rate to pyroscope.configure() as int for pyroscope-io - Replace print with verbose_proxy_logger (info/warning) - Document PYROSCOPE_SAMPLE_RATE in config_settings.md * Address Greptile PR feedback: Pyroscope optional, docs, tests, docstring - pyproject.toml: mark pyroscope-io as optional=true (proxy extra only) - Add docs/my-website/docs/proxy/pyroscope_profiling.md (fix broken sidebar link) - Add tests/test_litellm/proxy/test_pyroscope.py for _init_pyroscope() - proxy_server: fix _init_pyroscope docstring (required server/app name, sample rate as int) * Update litellm/proxy/proxy_server.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Several Gemini models (TTS, native-audio, robotics, gemma) were missing tpm/rpm values, causing test_get_model_info_gemini to fail. Added conservative default values (tpm=250000, rpm=10) for preview models. gemini-2.5-flash-preview-tts gets tpm=4000000, rpm=10. Co-authored-by: OpenClaw <openclaw@users.noreply.github.com>
…erriAI#21178) Co-authored-by: shin-bot-litellm <shin-bot-litellm@users.noreply.github.com>
- vertex_ai/gemini: fix TypedDict assignment via explicit dict cast - mcp_server: convert MutableMapping scope to dict for type safety - pass_through_endpoints: simplify custom_body logic to fix type narrowing - vector_store_endpoints: add Any annotation for dynamic hook return - responses transformation: use dict() for Reasoning and setattr for dynamic field - zscaler_ai_guard: add assert for api_base None check Co-authored-by: shin-bot-litellm <shin-bot-litellm@users.noreply.github.com>
* fix(ci): Fix ruff lint error - unused import
Remove unused 'cast' import in vertex_ai_ingestion.py (ruff F401)
* fix(ci): Fix E2E login button selector - use exact match
Login button selector now matches both 'Login' and 'Login with SSO',
causing strict mode violation. Use { exact: true } to match only 'Login'.
---------
Co-authored-by: OpenClaw <openclaw@users.noreply.github.com>
- vertex_ai/gemini/transformation.py: Fix TypedDict assignment via dict alias - mcp_server/server.py: Convert ASGI scope to dict for type compatibility - pass_through_endpoints.py: Add explicit Optional[dict] type annotation - vector_store_endpoints/endpoints.py: Add Any type for dynamic proxy hook - responses transformation.py: Use dict(Reasoning()) and setattr for compatibility - zscaler_ai_guard.py: Add assert for api_base nullability Co-authored-by: OpenClaw <openclaw@users.noreply.github.com>
…l execution (BerriAI#21177) * Add pipeline type definitions for guardrail pipelines PipelineStep, GuardrailPipeline, PipelineStepResult, PipelineExecutionResult with validation for actions (allow/block/next/modify_response) and modes. * Export pipeline types from policy_engine types package * Add optional pipeline field to Policy model * Add pipeline executor for sequential guardrail execution * Parse pipeline config in policy registry * Add pipeline validation in policy validator * Add pipeline resolution and managed guardrail tracking * Resolve pipelines and exclude managed guardrails in pre-call * Integrate pipeline execution into proxy pre_call_hook * Add test guardrails for pipeline E2E testing * Add example pipeline config YAML * Add unit tests for pipeline type definitions * Add unit tests for pipeline executor * Update litellm/proxy/policy_engine/pipeline_executor.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * Update litellm/proxy/utils.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Add pipeline type definitions for guardrail pipelines PipelineStep, GuardrailPipeline, PipelineStepResult, PipelineExecutionResult with validation for actions (allow/block/next/modify_response) and modes. * Export pipeline types from policy_engine types package * Add optional pipeline field to Policy model * Add pipeline executor for sequential guardrail execution * Parse pipeline config in policy registry * Add pipeline validation in policy validator * Add pipeline resolution and managed guardrail tracking * Resolve pipelines and exclude managed guardrails in pre-call * Integrate pipeline execution into proxy pre_call_hook * Add test guardrails for pipeline E2E testing * Add example pipeline config YAML * Add unit tests for pipeline type definitions * Add unit tests for pipeline executor * Add pipeline column to LiteLLM_PolicyTable schema * Add pipeline field to policy CRUD request/response types * Add pipeline support to policy DB CRUD operations * Add PipelineStep and GuardrailPipeline TypeScript types * Add Zapier-style pipeline flow builder UI component * Integrate pipeline flow builder with mode toggle in policy form * Add pipeline display section to policy info view * Add unit tests for pipeline in policy CRUD types * Refactor policy form to show mode picker first with icon cards * Add full-screen FlowBuilderPage component for pipeline editing * Wire up full-screen flow builder in PoliciesPanel with edit routing * Restyle flow builder to match dev-tool UI aesthetic * Restyle flow builder cards to match reference design * Update step card to expanded layout with stacked ON PASS / ON FAIL sections * Add end card to flow builder showing return to normal control flow * Add PipelineTestRequest type for test-pipeline endpoint * Export PipelineTestRequest from policy_engine types * Add POST /policies/test-pipeline endpoint * Add testPipelineCall networking function * Add PipelineStepResult and PipelineTestResult types * Add test pipeline panel to flow builder with run button and results display * Fix pipeline executor: inject guardrail name into metadata so should_run_guardrail allows execution * Update litellm/proxy/policy_engine/pipeline_executor.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * Update litellm/proxy/utils.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * Update litellm/proxy/policy_engine/policy_endpoints.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * Update litellm/proxy/policy_engine/pipeline_executor.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
…ctions
When system message content is a list of content blocks
(e.g. [{"type": "text", "text": "..."}]) instead of a plain string,
the responses API bridge was passing it through as a role: system
message in the input items. APIs like ChatGPT Codex reject this
with "System messages are not allowed".
This happens when requests come through the Anthropic /v1/messages
adapter, which converts system prompts into list-format content blocks
in the OpenAI chat completions format.
Fix: extract text from list content blocks and concatenate into the
instructions parameter, matching the existing behavior for string
system content.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile OverviewGreptile SummaryFixes a bug where system messages with list-format content blocks (e.g.
Confidence Score: 4/5
|
| Filename | Overview |
|---|---|
| litellm/completion_extras/litellm_responses_transformation/transformation.py | Replaces the catch-all else branch for system messages with an explicit elif isinstance(content, list) that extracts text from content blocks into instructions. The logic is correct but lacks test coverage and silently drops unrecognized content types. |
Flowchart
flowchart TD
A[Anthropic /v1/messages request<br/>with system prompt] --> B[Anthropic Adapter<br/>_add_system_message_to_messages]
B -->|string system| C[ChatCompletionSystemMessage<br/>content = string]
B -->|list system blocks| D["ChatCompletionSystemMessage<br/>content = [{type: text, text: ...}]"]
C --> E[Responses Bridge<br/>convert_chat_completion_messages_to_responses_api]
D --> E
E -->|isinstance str| F[instructions = content]
E -->|isinstance list NEW| G["Extract text from blocks<br/>join into instructions"]
E -->|else REMOVED| H["❌ Old: passed as role:system input item<br/>→ 'System messages not allowed' error"]
F --> I[Responses API Request<br/>instructions parameter set]
G --> I
I --> J[API call succeeds]
Last reviewed commit: 19ef616
| elif isinstance(content, list): | ||
| # Extract text from content blocks (e.g. [{"type": "text", "text": "..."}]) | ||
| text_parts = [] | ||
| for block in content: | ||
| if isinstance(block, dict) and block.get("type") == "text": | ||
| text_parts.append(block.get("text", "")) | ||
| elif isinstance(block, str): | ||
| text_parts.append(block) | ||
| extracted = " ".join(text_parts) | ||
| if instructions: | ||
| instructions = f"{instructions} {extracted}" | ||
| else: | ||
| instructions = extracted |
There was a problem hiding this comment.
Missing test coverage for new branch
This PR adds handling for list-format system content, but there are no corresponding unit tests. The existing test file (tests/test_litellm/completion_extras/litellm_responses_transformation/test_completion_extras_litellm_responses_transformation_transformation.py) has no tests for system message handling at all — neither for string content nor for this new list-content path.
Please add a test case that verifies:
- A system message with
content: [{"type": "text", "text": "Hello"}, {"type": "text", "text": "World"}]producesinstructions = "Hello World"and no system input items. - A system message with list content combined with another string system message properly concatenates into
instructions.
This would help prevent regressions and satisfy the PR template requirement of "Add at least 1 test in tests/litellm/".
Context Used: Rule from dashboard - What: Ensure that any PR claiming to fix an issue includes evidence that the issue is resolved, such... (source)
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
| elif isinstance(content, list): | ||
| # Extract text from content blocks (e.g. [{"type": "text", "text": "..."}]) | ||
| text_parts = [] | ||
| for block in content: | ||
| if isinstance(block, dict) and block.get("type") == "text": | ||
| text_parts.append(block.get("text", "")) | ||
| elif isinstance(block, str): | ||
| text_parts.append(block) | ||
| extracted = " ".join(text_parts) | ||
| if instructions: | ||
| instructions = f"{instructions} {extracted}" | ||
| else: | ||
| instructions = extracted |
There was a problem hiding this comment.
Silently drops non-str/list system content
The old else branch handled any non-string content type by passing it through as an input item. The new code only handles str and list — any other content type (e.g., a dict or unexpected type) is now silently ignored. While this is unlikely to occur given the AllMessageValues typing, consider adding a final else clause with a warning log for defensive robustness:
| elif isinstance(content, list): | |
| # Extract text from content blocks (e.g. [{"type": "text", "text": "..."}]) | |
| text_parts = [] | |
| for block in content: | |
| if isinstance(block, dict) and block.get("type") == "text": | |
| text_parts.append(block.get("text", "")) | |
| elif isinstance(block, str): | |
| text_parts.append(block) | |
| extracted = " ".join(text_parts) | |
| if instructions: | |
| instructions = f"{instructions} {extracted}" | |
| else: | |
| instructions = extracted | |
| elif isinstance(content, list): | |
| # Extract text from content blocks (e.g. [{"type": "text", "text": "..."}]) | |
| text_parts = [] | |
| for block in content: | |
| if isinstance(block, dict) and block.get("type") == "text": | |
| text_parts.append(block.get("text", "")) | |
| elif isinstance(block, str): | |
| text_parts.append(block) | |
| extracted = " ".join(text_parts) | |
| if instructions: | |
| instructions = f"{instructions} {extracted}" | |
| else: | |
| instructions = extracted | |
| else: | |
| verbose_logger.warning( | |
| f"Unexpected system message content type: {type(content)}" | |
| ) |
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
|
Note: All CI failures are pre-existing on
The |
Add three tests for convert_chat_completion_messages_to_responses_api: - String system content → instructions - List-format content blocks → instructions (the bug this PR fixes) - Multiple system messages (mixed string and list) concatenated
Address review feedback: add an else clause that logs a warning for any system content that is neither str nor list, rather than silently dropping it.
495ce34
into
BerriAI:litellm_oss_staging_02_14_2026
Summary
When system message content is a list of content blocks (e.g.
[{"type": "text", "text": "..."}]) instead of a plain string, the responses API bridge inconvert_chat_completion_messages_to_responses_api()was passing it through as arole: systemmessage in the input items. APIs like ChatGPT Codex reject this with"System messages are not allowed".This happens when requests arrive via the Anthropic
/v1/messagesadapter, which converts system prompts into list-format content blocks in the OpenAI chat completions format before the responses bridge processes them.Fix
Extract text from list content blocks and concatenate into the
instructionsparameter, matching the existing behavior for string system content. Theelsebranch that added list system content as input items is replaced with anelif isinstance(content, list)branch that joins text parts.How to reproduce
chatgpt/gpt-5.2-codex) withmode: "responses"/v1/messagesendpoint with a system messagerole: systemin input items"System messages are not allowed"Changes
litellm/completion_extras/litellm_responses_transformation/transformation.py: Replace theelsebranch (which appended list system content as input items) with anelif isinstance(content, list)branch that extracts text from content blocks intoinstructions