fix(responses-bridge): custom tool round-trip and allowlist preservation for Codex CLI by mateo-berri · Pull Request #32258 · BerriAI/litellm

mateo-berri · 2026-07-06T18:23:23Z

Relevant issues

Copy of #31571 by @duanhongyi, recreated on an internal branch so CircleCI can run. All credit for the original fix goes to @duanhongyi; the first commit preserves his authorship

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have added meaningful tests
My PR passes all CI/CD checks (e.g., lint, format, unit tests)
My PR's scope is as isolated as possible; it only solves 1 specific problem
I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

See the original PR #31571 for context; this PR carries the same commit, rebased onto the latest litellm_internal_staging, plus review-feedback commits

Type

🐛 Bug Fix

Changes

When the bridge routes /responses requests to providers that only support Chat Completions, tools with type: "custom" (for example Codex CLI's apply_patch) were silently dropped or treated as plain function tools. The model then returned a function_call output item that the client could not map back to a custom_tool_call, breaking the round-trip. The conversion also discarded allowed_callers, so a tool meant to be callable only by another tool could be invoked directly by the model

The new litellm/responses/litellm_completion_transformation/custom_tools.py module forward-converts custom tools to function tools (with the grammar embedded in the description so the model produces correctly formatted output), reverse-converts function_call responses back to custom_tool_call output items (unwrapping the {"content": ...} JSON envelope), and validates and preserves allowed_callers. transformation.py and streaming_iterator.py share this module so both the non-streaming and streaming paths emit custom_tool_call items when the call targets a tool that was originally custom

router.py now falls back to the source iterator's completed_response when the inner streaming generator is exhausted instead of logging a spurious "no completed_response" warning, and common_request_processing.py guards _apply_client_disconnect_metadata against a None target dict during streaming disconnects. CustomToolCallOutputItem is added in litellm/types/responses/main.py and registered in the ResponsesAPIResponse output item union

Follow-up commits address review feedback on the copy. From Greptile: convert_custom_tool_to_function_tool is now typed against Mapping/ChatCompletionToolParam with allowed_callers validated by a strict TypeAdapter (removing the two cast() calls that tripped the LIT006 ceiling in lint); dropping Responses-only tool types (computer_use, image_generation, namespace, shell) now logs a warning instead of being silent; output items are returned as Pydantic models instead of model_dump()ing every item to a dict; the request_data metadata paths that still used setdefault now use the same None-safe pattern as the logging_obj paths; and the unused build_custom_tool_call_item helper is removed. From Bugbot: payload recovery from input also triggers when arguments is an empty string, that recovery is scoped to custom_tool_call items so a plain function_call with empty arguments is never rewritten into a {"content": ...} envelope, and a function_call item with no arguments key now yields an empty arguments string instead of the literal string "None". From Veria: the Responses guardrail translation handler now extracts custom tool names too, so check_tools_allowlist (metadata.allowed_tools) rejects a disallowed tool declared as type: "custom" instead of letting the bridge convert it into a callable function tool

Tests cover the custom to function to custom_tool_call round-trip, allowed_callers preservation, the transformation integration, both client disconnect metadata fixes, allowlist enforcement for custom tools on the responses route, and the scoped input payload recovery including missing and empty arguments for both item types

…ion for Codex CLI Convert Responses API custom tools to Chat Completions function tools and map function_call responses back to custom_tool_call output items so Codex CLI gets the apply_patch round-trip it expects. Preserve and validate allowed_callers during the custom->function conversion so the Anthropic adapter's caller allowlist is not silently dropped, which would let a tool meant to be callable only by another tool be invoked directly by the model. Use modern type annotations (list/dict/set/X | None) throughout to keep the ruff strict budget within its ratcheted ceilings.

mateo-berri · 2026-07-06T18:23:53Z

@greptileai

Generated by Claude Code

codecov · 2026-07-06T18:28:56Z

Codecov Report

❌ Patch coverage is 92.17391% with 18 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...itellm_completion_transformation/transformation.py	82.29%	17 Missing ⚠️
.../litellm_completion_transformation/custom_tools.py	98.50%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

greptile-apps · 2026-07-06T18:30:44Z

Greptile Summary

This PR fixes the Responses → Chat Completions bridge so type: "custom" tools (e.g. Codex CLI's apply_patch) round-trip correctly. A new custom_tools module handles forward conversion (custom → function tool, grammar embedded in description) and reverse conversion (function_call response → custom_tool_call output item), and is shared by both the non-streaming transformation and the streaming iterator.

transform_responses_api_tools_to_chat_completion_tools now converts custom tools to function tools and drops Responses-API-only types with a warning log instead of silently passing them through.
extract_request_tool_names on the guardrail handler now includes type: "custom" tools, closing an allowlist bypass.
FallbackResponsesStreamWrapper.__anext__ copies source_iterator.completed_response on StopAsyncIteration, and _record_streaming_client_disconnect_if_needed uses explicit get+reassign to avoid propagating None metadata dicts.

Confidence Score: 5/5

Safe to merge; all changes are scoped to the responses bridge and carry comprehensive test coverage.

The custom tool round-trip, allowlist enforcement, streaming latch, and disconnect-metadata fixes are all well-implemented and well-tested. The logic is consistent across the streaming and non-streaming paths, the security-relevant allowlist enforcement is correct, and the multi-turn history reconstruction for custom_tool_call items is properly scoped to avoid corrupting plain function_call items.

litellm/responses/litellm_completion_transformation/custom_tools.py — the allowed_callers placement in ChatCompletionToolParam could affect strict OpenAI-compatible providers, but this is limited to the unusual case where a custom tool carries allowed_callers.

Important Files Changed

Filename	Overview
litellm/responses/litellm_completion_transformation/custom_tools.py	New module for forward/reverse conversion between Responses API custom tools and Chat Completions function tools.
litellm/responses/litellm_completion_transformation/transformation.py	Extended to convert custom tools, drop unsupported types with warning, and recover custom_tool_call payloads in multi-turn history.
litellm/responses/litellm_completion_transformation/streaming_iterator.py	Uses build_tool_call_item_kwargs for custom_tool_call streaming events and latches completed_response.
litellm/router.py	FallbackResponsesStreamWrapper now copies source_iterator.completed_response on StopAsyncIteration.
litellm/proxy/common_request_processing.py	Guards _apply_client_disconnect_metadata against None and uses explicit get+reassign pattern.
litellm/llms/openai/responses/guardrail_translation/handler.py	extract_request_tool_names includes type: custom tools for allowlist enforcement.
litellm/types/responses/main.py	Adds CustomToolCallOutputItem Pydantic model.
litellm/types/llms/openai.py	Registers CustomToolCallOutputItem in ResponsesAPIResponse union and adds allowed_callers to ChatCompletionToolParam.

_{Reviews (6): Last reviewed commit: "fix(responses-bridge): default missing f..." | Re-trigger Greptile}

greptile-apps · 2026-07-06T18:31:22Z

Greptile Summary

This PR backports a fix (originally #31571 by @duanhongyi) for the custom tool round-trip in the /responses → Chat Completions bridge, targeting Codex CLI's apply_patch and similar type: \"custom\" tools. A new custom_tools.py module centralises the forward (custom→function) and reverse (function_call→custom_tool_call) conversions, and CustomToolCallOutputItem is added to the type system and output union.

Custom tool round-trip: Custom tools are converted to function tools (grammar embedded in description), and function_call responses are unwrapped back to custom_tool_call items with raw input instead of JSON-wrapped arguments. Both the streaming and non-streaming paths share the same helper.
Bug fixes: _apply_client_disconnect_metadata now guards against None metadata dicts; FallbackResponsesStreamWrapper falls back to source_iterator.completed_response on StopAsyncIteration; computer_use, image_generation, namespace, and shell tools are dropped rather than passed through verbatim.
Type modernisation: Union[A, B] / Optional[X] / List[...] annotations replaced with modern A | B / X | None / list[...] syntax throughout the changed files.

Confidence Score: 4/5

The core custom tool round-trip logic is well-tested and the three targeted bug fixes are low-risk; two non-blocking gaps in observability and return-type clarity are worth addressing before merge.

The custom tool conversion, streaming latch, and client-disconnect fixes are correct and covered by new tests. Two concerns keep this from a fully clean assessment: unsupported tool types are silently discarded with no warning log, and model_dump() on line 1693 converts every pre-existing output item to a plain dict, broader than the new CustomToolCallOutputItem alone required.

transformation.py (silent tool drop at lines 1125-1138 and broad model_dump at line 1693); custom_tools.py (unused build_custom_tool_call_item)

Important Files Changed

Filename	Overview
litellm/responses/litellm_completion_transformation/custom_tools.py	New module implementing custom→function tool conversion and function_call→custom_tool_call reverse conversion; `build_custom_tool_call_item` is defined but never called from production code
litellm/responses/litellm_completion_transformation/transformation.py	Custom tool conversion integrated into the non-streaming path; `model_dump()` now applied to all output items; `computer_use`/`image_generation`/`namespace`/`shell` tools silently dropped without a warning
litellm/responses/litellm_completion_transformation/streaming_iterator.py	Streaming path updated to emit `custom_tool_call` items via the shared helper; `completed_response` latched on the response.completed event; mostly type-annotation modernisation
litellm/proxy/common_request_processing.py	Correctly fixes crash when metadata is explicitly `None` during a streaming client disconnect
litellm/router.py	FallbackResponsesStreamWrapper falls back to source_iterator.completed_response on StopAsyncIteration
litellm/types/responses/main.py	Adds `CustomToolCallOutputItem` Pydantic model
litellm/types/llms/openai.py	Adds `CustomToolCallOutputItem` to the `ResponsesAPIResponse.output` union type
tests/test_litellm/responses/test_custom_tool_call.py	New mock-only test file covering the custom tool round-trip
tests/test_litellm/responses/litellm_completion_transformation/test_litellm_completion_responses.py	Existing computer_use test expectation updated to reflect drop behaviour; new tests cover custom tool conversion and `allowed_callers` preservation
tests/test_litellm/proxy/test_common_request_processing.py	New mock-only tests verify the None-metadata guard

_{Reviews (2): Last reviewed commit: "fix(responses-bridge): custom tool round..." | Re-trigger Greptile}

Type convert_custom_tool_to_function_tool against Mapping/ChatCompletionToolParam and validate allowed_callers with a strict TypeAdapter so the two new cast() calls that tripped the LIT006 ceiling are gone. Warn when dropping Responses-only tool types (computer_use, image_generation, namespace, shell) instead of discarding them silently. Return output items as Pydantic models instead of model_dump()ing every item to a dict, matching the declared return type. Apply the same None-safe metadata pattern to the request_data paths that still used setdefault, and drop the unused build_custom_tool_call_item helper.

mateo-berri · 2026-07-06T18:56:15Z

@greptileai

Generated by Claude Code

mateo-berri · 2026-07-06T19:08:24Z

bugbot run

Generated by Claude Code

veria-ai · 2026-07-06T19:10:12Z

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

cursor

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

✅ Fixed: Empty arguments drop custom input
- Changed the recovery guard from is None to a falsy check so an empty arguments string still falls back to reconstructing the payload from input.

_{You can send follow-ups to the cloud agent here.}

CLAassistant · 2026-07-06T19:22:29Z

All committers have signed the CLA.

…ponses route The Responses guardrail translation handler only extracted function and mcp tool names, so a key or team restricted by metadata.allowed_tools could invoke a disallowed tool by declaring it with type custom now that the bridge converts custom tools into callable Chat Completions function tools. Extract custom tool names through the same path so check_tools_allowlist rejects them.

mateo-berri · 2026-07-06T22:11:23Z

@greptileai

Generated by Claude Code

mateo-berri · 2026-07-06T22:15:15Z

bugbot run

Generated by Claude Code

cursor

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

✅ Fixed: Empty arguments reuse input field
- Gated the input-based arguments rebuild on type == "custom_tool_call" so regular function_call items with empty-string arguments are no longer wrapped into a {"content": ...} envelope.

_{You can send follow-ups to the cloud agent here.}

…ll items Recovering tool arguments from the input field on any falsy arguments value made plain function_call input items with empty arguments and a stray input key get rewritten into a {"content": ...} envelope, corrupting multi-turn replay for normal function tools. Gate the recovery on the item type so it only applies to custom_tool_call items, which are the ones that store their payload in input.

mateo-berri · 2026-07-06T22:27:55Z

@greptileai

Generated by Claude Code

mateo-berri · 2026-07-06T22:36:28Z

bugbot run

Generated by Claude Code

cursor

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

✅ Fixed: Missing arguments become literal None
- Changed str(raw_arguments) to str(raw_arguments or "") so a missing/None arguments key produces an empty string instead of the literal "None".

_{You can send follow-ups to the cloud agent here.}

…ty string With input recovery scoped to custom_tool_call items, a plain function_call input item without an arguments key left raw_arguments as None and the downstream str() turned it into the literal string None. Coerce to an empty string instead, matching the pre-bridge behavior.

mateo-berri · 2026-07-06T22:43:46Z

@greptileai

Generated by Claude Code

mateo-berri · 2026-07-06T22:49:44Z

bugbot run

Generated by Claude Code

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 87347a6. Configure here.}

mateo-berri marked this pull request as ready for review July 6, 2026 18:23

greptile-apps Bot reviewed Jul 6, 2026

View reviewed changes

Comment thread litellm/responses/litellm_completion_transformation/transformation.py

Comment thread ...est_litellm/responses/litellm_completion_transformation/test_litellm_completion_responses.py

greptile-apps Bot reviewed Jul 6, 2026

View reviewed changes

Comment thread litellm/responses/litellm_completion_transformation/custom_tools.py Outdated

Comment thread litellm/responses/litellm_completion_transformation/transformation.py

Comment thread litellm/responses/litellm_completion_transformation/transformation.py Outdated

veria-ai Bot reviewed Jul 6, 2026

View reviewed changes

Comment thread litellm/responses/litellm_completion_transformation/transformation.py

cursor Bot reviewed Jul 6, 2026

View reviewed changes

Comment thread litellm/responses/litellm_completion_transformation/transformation.py

mateo-berri added 2 commits July 6, 2026 22:02

fix(responses-bridge): recover custom tool input when arguments is empty

ac112f8

mateo-berri force-pushed the litellm_fix_codex_compatibility branch from 910c211 to 65db265 Compare July 6, 2026 22:10

cursor Bot reviewed Jul 6, 2026

View reviewed changes

Comment thread litellm/responses/litellm_completion_transformation/transformation.py Outdated

mateo-berri force-pushed the litellm_fix_codex_compatibility branch from 4a5b764 to 07ba468 Compare July 6, 2026 22:27

cursor Bot reviewed Jul 6, 2026

View reviewed changes

Comment thread litellm/responses/litellm_completion_transformation/transformation.py

mateo-berri force-pushed the litellm_fix_codex_compatibility branch from 5f87711 to 87347a6 Compare July 6, 2026 22:43

cursor Bot reviewed Jul 6, 2026

View reviewed changes

yucheng-berri approved these changes Jul 7, 2026

View reviewed changes

mateo-berri merged commit 2f0cdb3 into litellm_internal_staging Jul 7, 2026
129 of 130 checks passed

mateo-berri deleted the litellm_fix_codex_compatibility branch July 7, 2026 00:34

This was referenced Jul 13, 2026

fix(proxy): guard _apply_client_disconnect_metadata against None metadata #32751

Closed

TypeError: 'NoneType' object does not support item assignment during streaming client disconnect cleanup (metadata=None) #32750

Closed

devin-ai-integration Bot mentioned this pull request Jul 17, 2026

fix(responses-bridge): drop Codex tool_search/local_shell tools for chat-only providers #33783

Open

5 tasks

Mattchewone mentioned this pull request Jul 20, 2026

chore: sync upstream v1.93.0 bitovi/litellm#14

Merged

mayank-affirm mentioned this pull request Jul 23, 2026

fix(responses): preserve custom tools through guardrails #34361

Draft

5 tasks

devin-ai-integration Bot mentioned this pull request Aug 5, 2026

fix(responses): forward allowed_openai_params through the chat completions bridge #35885

Merged

5 tasks

Uh oh!

Uh oh!

Conversation

mateo-berri commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Relevant issues

Linear ticket

Pre-Submission checklist

Delays in PR merge?

Screenshots / Proof of Fix

Type

Changes

Uh oh!

mateo-berri commented Jul 6, 2026

Uh oh!

codecov Bot commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

greptile-apps Bot commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Uh oh!

Uh oh!

Uh oh!

greptile-apps Bot commented Jul 6, 2026

Greptile Summary

Confidence Score: 4/5

Important Files Changed

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mateo-berri commented Jul 6, 2026

Uh oh!

mateo-berri commented Jul 6, 2026

Uh oh!

Uh oh!

veria-ai Bot commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR overview

Security review

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

CLAassistant commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mateo-berri commented Jul 6, 2026

Uh oh!

mateo-berri commented Jul 6, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mateo-berri commented Jul 6, 2026

Uh oh!

mateo-berri commented Jul 6, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mateo-berri commented Jul 6, 2026

Uh oh!

mateo-berri commented Jul 6, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

mateo-berri commented Jul 6, 2026 •

edited

Loading

codecov Bot commented Jul 6, 2026 •

edited

Loading

greptile-apps Bot commented Jul 6, 2026 •

edited

Loading

veria-ai Bot commented Jul 6, 2026 •

edited

Loading

cursor Bot left a comment •

edited

Loading

CLAassistant commented Jul 6, 2026 •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading