feat(sandbox): code interpreter interceptor on the Responses API - #30905
Merged
krrish-berri-2 merged 20 commits intoJun 21, 2026
Merged
Conversation
Route OpenAI's code interpreter to a configured sandbox (e2b) instead of OpenAI's container, with no client change. A client calls /v1/responses with a code_interpreter tool; the interceptor converts it to a function tool so the model emits the code, runs that code in the sandbox via the phase 1 primitive, feeds the result back, and lets the agentic loop continue. Reuses the existing agentic-loop hooks (no new hook methods). The anthropic agentic caller _call_agentic_completion_hooks gains an api_surface argument and a responses execute path (_execute_responses_agentic_plan re-calls aresponses); the responses handler invokes it after transforming the response. Web search and compression interceptors are untouched. Adds an api_base passthrough to the sandbox SDK and a sandbox_tools registry the proxy parses, so the interceptor resolves a named tool to provider/key/base. v0 limitation: no file upload or download yet; stdout and inline results flow back, attaching input files and downloading produced files do not.
|
|
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a83c4df111
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…s OpenAI The native OpenAI Responses code interpreter returns a code_interpreter_call output item (id, type, status, code, container_id, outputs) alongside the message. The interceptor now re-injects an equivalent item via async_post_agentic_loop_response_hook so a client gets the same response shape whether the code ran in OpenAI's container or the sandbox: build_plan records the executed code and the container id per call, and the post hook inserts the code_interpreter_call before the message in the final response output.
Contributor
Greptile SummaryThis PR adds a
Confidence Score: 5/5Safe to merge; the security gating (server-minted markers, all control fields stripped at the proxy boundary) is solid and well-tested. The core agentic-loop wiring, sandbox lifecycle, and request-boundary stripping are all correct and comprehensively covered by 116 passing unit tests with dependency injection. The only concerns are an overly broad tool_choice name match that can only misfire when a user simultaneously has both a native code_interpreter tool and a custom function tool of the same name, and a pruner double-delete that is already caught by the exception handler. handler.py — the two findings both live there, in _tool_choice_targets_code_interpreter and _prune_expired_cache.
|
| Filename | Overview |
|---|---|
| litellm/integrations/code_interpreter_interception/handler.py | New interceptor logger: converts code_interpreter tool to function tool, runs code in e2b sandbox, re-injects code_interpreter_call output items. Minor edge case in tool_choice rewriting (see comment). |
| litellm/llms/custom_httpx/llm_http_handler.py | Adds _execute_responses_agentic_plan and _wrap_responses_response_as_fake_stream; threads api_surface through _call_agentic_completion_hooks; guards websearch stream conversion to anthropic_messages surface only. Clean integration into the existing agentic loop machinery. |
| litellm/sandbox/sandbox_tools.py | New registry for sandbox tools; atomic swap on reload, env-var secret resolution, malformed-entry filtering. Correct. |
| litellm/proxy/litellm_pre_call_utils.py | Adds all six agentic-loop and code-interpreter control fields to _UNTRUSTED_ROOT_CONTROL_FIELDS, preventing client forgery. Parametrized test covers all new fields. |
| litellm/proxy/proxy_server.py | Registers sandbox_tools from config unconditionally (empty-list fallback clears stale credentials on reload). One-liner addition in the config load path. |
Reviews (16): Last reviewed commit: "fix(code-interpreter): delete cached san..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![veria-ai[bot]](https://avatars.githubusercontent.com/in/1723776?s=60&v=4){kind=small}
Contributor
PR overviewAll previously flagged issues have been addressed. No open security concerns remain on this pull request. Security reviewNo open security issues remain on this pull request. Fixed/addressed: 7 · PR risk: 0/10 |
A stream:true /v1/responses request with code_interpreter previously broke, because the agentic loop only runs on the non-streaming responses path. The interceptor now forces stream=False in the pre-call hook (so the loop runs in the sandbox) and the responses handler wraps the completed response back into a synthetic stream via MockResponsesAPIStreamingIterator, so the caller still gets SSE. The follow-up call and nested wrapping are guarded by stripping the converted-stream flag from the follow-up request and only wrapping at the outermost call (agentic loop depth 0).
…tisfy UP006 budget
…en registry Gate the agentic loop on a server-set interception marker and re-check provider scope so an authenticated caller cannot trigger sandbox code execution by naming their own function tool litellm_code_execution; the marker is stripped from client requests at the proxy boundary and only set when the pre-call hook actually converts a native code_interpreter tool. Delete the sandbox once the final response is assembled instead of leaking it until its own timeout, and prune expired cache entries by deleting their containers too. Resolve sandbox params once at create time and reuse them for run and delete. Clear the sandbox-tool registry before re-registering so stale tools do not survive a config reload.
Contributor
Author
|
@greptileai review |
Contributor
Author
|
@greptileai review |
1 similar comment
Contributor
Author
|
@greptileai review |
…t tool-call paths
Contributor
Author
|
@greptileai review |
… the function tool
Contributor
Author
|
@greptileai review |
…g, and secret lookup
…ute tool-call detection
Contributor
Author
|
@greptileai review |
…erted_stream A client could inject the converted-stream marker to force the completed response to be re-wrapped as a synthetic SSE stream it never requested. Add it to the untrusted root control fields alongside the other agentic loop markers so the proxy strips it at the request boundary.
Contributor
Author
|
@greptileai review |
Contributor
Author
|
@greptileai review |
…ar registry on tool removal Key the per-request sandbox cache on a server-minted random token instead of the caller-controlled litellm_call_id (sourced from the x-litellm-call-id header). Two concurrent requests that send a colliding call id can no longer share a sandbox container and read each other's code or files. The token is minted in the pre-call hook when interception activates, stripped from client requests at the proxy boundary, and survives the server-driven followups so a single request still reuses one sandbox across the agentic loop. Register sandbox tools unconditionally with an empty-list fallback so a config reload that removes sandbox_tools clears the previously registered credentials instead of leaving them resolvable in the process.
…utputs
Strip max_agentic_loops at the proxy request boundary so an authenticated
caller cannot raise the agentic-loop ceiling to drive many upstream model
calls and sandbox executions from a single request; the loop stays bounded
by the server default.
Populate the re-injected code_interpreter_call.outputs with an OpenAI-shaped
logs array ([{"type": "logs", "logs": stdout}], or [] when there is no
stdout) instead of None, so clients that iterate over outputs or validate the
response through the OpenAI SDK's Pydantic model do not break.
Contributor
Author
|
@greptileai review |
krrish-berri-2
enabled auto-merge (squash)
June 21, 2026 02:58
…g registration A sandbox_tools entry missing sandbox_tool_name (or not a dict) raised KeyError in register_sandbox_tools, which runs at proxy startup and hot-reload; one bad YAML entry would abort the whole registration and leave the registry empty or cleared, failing every later code-interpreter request. Malformed entries are now skipped with a warning and the well-formed entries still register.
Contributor
Author
|
@greptileai review |
Contributor
Author
krrish-berri-2
pushed a commit
to BerriAI/litellm-docs
that referenced
this pull request
Jun 21, 2026
Documents BerriAI/litellm#30905: how a client can call OpenAI's code interpreter through /v1/responses and have the code run in the configured sandbox (e2b) instead of OpenAI's container, with no client-side change. Covers the sandbox_tools registry and code_interpreter_interception callback, the curl shape, response-shape parity with native code_interpreter_call (id/type/status/code/container_id/outputs as an OpenAI-shaped logs array), streaming via the synthetic SSE wrapper, forced tool_choice rewrite, sandbox lifecycle and per-request isolation on a server-minted token, the stripped control fields that prevent client forgery, hot-reload behaviour, and the v0 limitation that file upload and download are not yet supported. Also notes the new api_base passthrough on the SDK entrypoints.
4 tasks
mateo-berri
approved these changes
Jun 21, 2026
Contributor
Author
|
@greptileai review |
…erun fails The follow-up aresponses call in _execute_responses_agentic_plan only ran async_post_agentic_loop_response_hook (which deletes the cached sandbox) on success. A loop safety abort or upstream error left the sandbox cached and running until the 15 minute prune TTL. Add an async_agentic_loop_cleanup_hook that runs in a finally around the rerun; the code interpreter handler deletes its sandbox idempotently so the success path and the finally path never double-delete
Contributor
Author
|
@greptileai review |
Contributor
Author
|
@veria-ai review |
…06 budget A sandbox_tools entry with a name but no sandbox_provider was registered and later failed at acreate_sandbox(provider=None) with a cryptic runtime error. _iter_valid_tools now warns and skips it, mirroring the missing-name path, so a misconfiguration surfaces as a clear startup warning. Annotate the new async_agentic_loop_cleanup_hook kwargs as the builtin dict so the strict-rule budget gate (UP006) stays under its ceiling
Contributor
Author
|
@greptileai review |
_get_or_create_container caches the sandbox before _run_tool_call executes the model's code. If execution raised before a plan was returned, for example when E2B aborts output over its cap, neither the post-response hook nor the cleanup hook ever received a plan and the paid container leaked until the prune TTL. Wrap the tool-execution loop so any failure deletes the cached sandbox key before re-raising. Idempotent with the existing finally-path cleanup so unwinding never double-deletes
Contributor
Author
|
@greptileai review |
krrish-berri-2
merged commit Jun 21, 2026
84c1414
into
litellm_internal_staging
122 of 124 checks passed
Contributor
Author
|
@greptileai review |
krrish-berri-2
added a commit
to BerriAI/litellm-docs
that referenced
this pull request
Jun 21, 2026
* docs(sandbox): document e2b code execution primitive Documents the SDK surface added in BerriAI/litellm#30898: acode_interpreter_tool for ephemeral runs and the acreate_sandbox / arun_code / adelete_sandbox lifecycle. Covers the e2b backend setup, parameters, the CodeExecutionResult passthrough shape, and what is intentionally out of scope for this phase (proxy endpoints, OpenAI container interceptor, gpt-5 routing, container reuse, file IO, OpenSandbox, Daytona). Adds a sidebar entry under LiteLLM Python SDK > SDK Functions. * docs(sandbox): add Responses API code interpreter interceptor section Documents BerriAI/litellm#30905: how a client can call OpenAI's code interpreter through /v1/responses and have the code run in the configured sandbox (e2b) instead of OpenAI's container, with no client-side change. Covers the sandbox_tools registry and code_interpreter_interception callback, the curl shape, response-shape parity with native code_interpreter_call (id/type/status/code/container_id/outputs as an OpenAI-shaped logs array), streaming via the synthetic SSE wrapper, forced tool_choice rewrite, sandbox lifecycle and per-request isolation on a server-minted token, the stripped control fields that prevent client forgery, hot-reload behaviour, and the v0 limitation that file upload and download are not yet supported. Also notes the new api_base passthrough on the SDK entrypoints. * docs(sandbox): cross-link the interceptor from the Code Interpreter guide A proxy user looking for code interpreter lands on guides/code_interpreter, not the SDK sandbox page, so the interceptor was effectively undiscoverable from the proxy side. Adds a tip in the Code Interpreter guide pointing at the new interceptor section, and a second sidebar entry under Tool Calling labelled 'Code Interpreter Sandbox Interception' next to the existing Web Search Interception entry, mirroring how websearch_interception is exposed. * docs(sandbox): turn the proxy section into a step-by-step tutorial The interceptor section had the config and curl but no narrative walking a proxy user from a clean checkout to a working call. Restructures into five numbered steps (get the e2b key, write config.yaml with a model_list plus sandbox_tools registry, start the proxy, call /v1/responses via curl or the OpenAI SDK, verify the code ran in e2b via the cntr_-wrapped container_id and the e2b dashboard sandbox count). Calls out enabled_providers as the safety gate so flipping the callback on does not silently change behaviour for other providers. * docs(sandbox): tighten the tutorial section * docs(sandbox): collapse the proxy section notes into prose --------- Co-authored-by: Krrish Dholakia <krrishdholakia@berri.ai>
Contributor
|
Greptile encountered an error while reviewing this PR. Please reach out to support@greptile.com for assistance. |
3 tasks
fzowl
pushed a commit
to fzowl/litellm
that referenced
this pull request
Jun 24, 2026
…riAI#30905) * feat(sandbox): code interpreter interceptor on the Responses API Route OpenAI's code interpreter to a configured sandbox (e2b) instead of OpenAI's container, with no client change. A client calls /v1/responses with a code_interpreter tool; the interceptor converts it to a function tool so the model emits the code, runs that code in the sandbox via the phase 1 primitive, feeds the result back, and lets the agentic loop continue. Reuses the existing agentic-loop hooks (no new hook methods). The anthropic agentic caller _call_agentic_completion_hooks gains an api_surface argument and a responses execute path (_execute_responses_agentic_plan re-calls aresponses); the responses handler invokes it after transforming the response. Web search and compression interceptors are untouched. Adds an api_base passthrough to the sandbox SDK and a sandbox_tools registry the proxy parses, so the interceptor resolves a named tool to provider/key/base. v0 limitation: no file upload or download yet; stdout and inline results flow back, attaching input files and downloading produced files do not. * feat(sandbox): re-inject code_interpreter_call so the response matches OpenAI The native OpenAI Responses code interpreter returns a code_interpreter_call output item (id, type, status, code, container_id, outputs) alongside the message. The interceptor now re-injects an equivalent item via async_post_agentic_loop_response_hook so a client gets the same response shape whether the code ran in OpenAI's container or the sandbox: build_plan records the executed code and the container id per call, and the post hook inserts the code_interpreter_call before the message in the final response output. * feat(sandbox): support streaming for the code interpreter interceptor A stream:true /v1/responses request with code_interpreter previously broke, because the agentic loop only runs on the non-streaming responses path. The interceptor now forces stream=False in the pre-call hook (so the loop runs in the sandbox) and the responses handler wraps the completed response back into a synthetic stream via MockResponsesAPIStreamingIterator, so the caller still gets SSE. The follow-up call and nested wrapping are guarded by stripping the converted-stream flag from the follow-up request and only wrapping at the outermost call (agentic loop depth 0). * fix(lint): use builtin generics in code interpreter interceptor to satisfy UP006 budget * fix(code-interpreter): gate sandbox execution, delete sandboxes, harden registry Gate the agentic loop on a server-set interception marker and re-check provider scope so an authenticated caller cannot trigger sandbox code execution by naming their own function tool litellm_code_execution; the marker is stripped from client requests at the proxy boundary and only set when the pre-call hook actually converts a native code_interpreter tool. Delete the sandbox once the final response is assembled instead of leaking it until its own timeout, and prune expired cache entries by deleting their containers too. Resolve sandbox params once at create time and reuse them for run and delete. Clear the sandbox-tool registry before re-registering so stale tools do not survive a config reload. * fix(lint): use PEP 604 X | None unions to satisfy UP045 budget * test(code-interpreter): cover execution-error and unparseable-argument tool-call paths * fix(code-interpreter): rewrite forced code_interpreter tool_choice to the function tool * test(sandbox): cover sandbox-tool registry resolution, reload clearing, and secret lookup * test(code-interpreter): cover dict-shaped responses and object-attribute tool-call detection * fix(proxy): strip client-supplied _code_interpreter_interception_converted_stream A client could inject the converted-stream marker to force the completed response to be re-wrapped as a synthetic SSE stream it never requested. Add it to the untrusted root control fields alongside the other agentic loop markers so the proxy strips it at the request boundary. * fix(code-interpreter): isolate sandboxes by server-minted key and clear registry on tool removal Key the per-request sandbox cache on a server-minted random token instead of the caller-controlled litellm_call_id (sourced from the x-litellm-call-id header). Two concurrent requests that send a colliding call id can no longer share a sandbox container and read each other's code or files. The token is minted in the pre-call hook when interception activates, stripped from client requests at the proxy boundary, and survives the server-driven followups so a single request still reuses one sandbox across the agentic loop. Register sandbox tools unconditionally with an empty-list fallback so a config reload that removes sandbox_tools clears the previously registered credentials instead of leaving them resolvable in the process. * refactor(sandbox): swap the tool registry atomically on reload Build the new registry and rebind it in one assignment instead of clearing then repopulating in place, so a concurrent resolve_sandbox_tool can never observe a transiently empty or half-populated registry during a config reload. clear_sandbox_tools now delegates to register_sandbox_tools([]). * fix(code-interpreter): cap caller loop limit and emit OpenAI-shaped outputs Strip max_agentic_loops at the proxy request boundary so an authenticated caller cannot raise the agentic-loop ceiling to drive many upstream model calls and sandbox executions from a single request; the loop stays bounded by the server default. Populate the re-injected code_interpreter_call.outputs with an OpenAI-shaped logs array ([{"type": "logs", "logs": stdout}], or [] when there is no stdout) instead of None, so clients that iterate over outputs or validate the response through the OpenAI SDK's Pydantic model do not break.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Relevant issues
Part of #30891 (phase 2: the code interpreter interceptor). Builds on #30898 (the e2b execution primitive).
Linear ticket
n/a
Pre-Submission checklist
make test-unitType
🆕 New Feature
Changes
A client calls OpenAI's code interpreter through litellm's Responses API and the code runs in a configured sandbox (e2b) instead of OpenAI's container, with no change to the request.
OpenAI runs code interpreter server-side, so a proxy cannot intercept the execution. So, the same way the web search and prompt compression interceptors work, the pre-call hook converts the native
code_interpretertool into alitellm_code_executionfunction tool; the model then emits the code as a tool call; the interceptor runs that code in the sandbox via the phase 1 primitive and feeds afunction_call_outputback; litellm's agentic loop continues until the model stops calling the tool.This reuses the existing agentic-loop hooks rather than adding new ones. The anthropic agentic caller
_call_agentic_completion_hooksgains anapi_surfaceargument and a responses execute branch (_execute_responses_agentic_plan, which re-callslitellm.aresponseswith the patched input); the responses handler invokes it aftertransform_response_api_response. The web search and compression interceptors are untouched, and their tests are the regression guard for the shared caller.The interceptor is
CodeInterpreterInterceptionLogger, modeled oncompression_interception/handler.py, with per-request container reuse keyed bylitellm_call_id. Alias resolution lives in the interceptor, not the SDK: the proxy parses a top-levelsandbox_toolsregistry and the interceptor resolvessandbox_tool_nameto a provider, key, and base. The SDK gains anapi_basepassthrough so a self-hosted sandbox can be pointed at a cluster URL later.Enable it with:
v0 limitation, called out in the docs: no file upload or download yet. stdout and inline results flow back; attaching input files and downloading produced files are not supported. Out of scope and tracked on the ticket: the OpenAI-compatible
/v1/containersmanagement endpoints, and the OpenSandbox and Daytona backends (parallel PR #3).Tests
tests/test_litellm/integrations/code_interpreter_interception/test_handler.pyand the api_base tests intests/test_litellm/sandbox/test_e2b_sandbox.py, all dependency-injected (a fake sandbox config, no monkeypatching). They cover the native-to-function tool conversion (and that it only fires on responses call types), detecting the function call, running the code and feeding the output back inbuild_plan, container reuse across two calls with the same call id, andapi_baseoverriding the default host.tests/test_litellm/sandbox/test_sandbox_tools.pycovers the registry itself: atomic swap on reload, clearing stale credentials when a tool is removed, env-var secret resolution, and skipping malformedsandbox_toolsentries (missingsandbox_tool_nameor not a dict) so one bad config line cannot crash registration at startup or hot-reload. The web search and compression interception suites stay green as the regression guard for the shared agentic caller.Screenshots / Proof of Fix
Live proxy on a port the CLI selected (11881 here), started with the example config and real
OPENAI_API_KEY+E2B_API_KEY. curl to/v1/responseswithgpt-5and acode_interpretertool:Result, with an e2b sandbox count taken immediately before and after the call:
The answer 30030 is correct (235711*13). The response contains no native
code_interpreter_callitem, which means OpenAI did not run the code; and the e2b running-sandbox count went from 2 to 3, which means the code executed in e2b. Real model, real sandbox, real money.Update: response format parity
The native OpenAI code interpreter returns a
code_interpreter_calloutput item next to the message. Earlier this PR returned only the message, so the response shape did not match. The interceptor now re-injects an equivalent item viaasync_post_agentic_loop_response_hook, so a client gets the same shape whether the code ran in OpenAI's container or the sandbox.OpenAI native (interceptor off):
Through our interceptor (e2b):
Same keys, real executed code,
status: completed. Thecontainer_idis litellm's standard responses container-id encoding wrapping the e2b sandbox id, so the client gets a stablecntr_id like OpenAI's. Three unit tests guard this: build_plan records the call, the post hook injects it before the message with exactly OpenAI's keys, and it is a no-op when nothing ran.Update: streaming
stream: trueis now supported. The pre-call hook forcesstream=Falseso the agentic loop runs in the sandbox, and the responses handler wraps the completed response back into a synthetic stream (MockResponsesAPIStreamingIterator), so the caller still gets SSE. Nested/double wrapping is guarded by stripping the converted-stream flag from the follow-up request and only wrapping at the outermost call (agentic loop depth 0).Live streaming run (gpt-5 + code_interpreter, e2b), assembling the answer from
output_text.deltaevents and reading theresponse.completedevent:Real SSE deltas, the completed event matches OpenAI's
[code_interpreter_call, message]shape, and the code ran in e2b (sandbox count +1). A unit test asserts the pre-call hook flipsstreamto False and sets the converted-stream flag.Test totals after streaming: 116 passed, 2 skipped (interceptor, sandbox, plus the web search and compression regression guard).
Update: execution gating and sandbox lifecycle
Execution is gated on a server-set marker so a caller cannot trigger sandbox code execution by defining their own function tool named
litellm_code_execution. The pre-call hook sets_code_interpreter_interception_activeonly when it actually converts a nativecode_interpretertool, andasync_should_run_agentic_looprefuses to run without that marker while re-checkingenabled_providersagainst the request's provider. The marker is un-forgeable from a client request: the proxy strips_code_interpreter_interception_activeand the agentic-loop control fields_agentic_loop_depth/_agentic_loop_fingerprintsat the request boundary, and the pre-call hook clears any client-supplied value on the initial request. It is preserved only on server-driven followups (_agentic_loop_depth > 0) so multi-round execution keeps working.Sandboxes are deleted once the final response is assembled rather than left running until their own timeout.
async_post_agentic_loop_response_hookpops the container from the per-call_id cache and deletes it; deletion is idempotent so the unwinding loop levels do not double-delete, and the TTL prune now deletes the underlying container for any orphaned entry instead of only dropping the cache key. Sandbox params are resolved once at create time and reused for run and delete, so a registry clear between create and run can no longer create-then-fail, andregister_sandbox_toolsclears the registry before re-registering so a tool removed from the config does not survive a hot reload.New regression tests cover the gate refusing without the marker, the provider re-check, the proxy stripping the forged control fields, the marker surviving server followups, sandbox deletion after the loop, idempotent deletion across loop levels, and single-resolution of registry params.
Update: forced tool_choice parity
A request can force the hosted tool with
tool_choice({"type":"code_interpreter"}or ahosted_toolwhose name iscode_interpreter). Since the pre-call hook swaps the nativecode_interpretertool for thelitellm_code_executionfunction tool, a forcedtool_choiceleft as-is would reference a tool that no longer exists and the provider would reject the request. The hook now rewrites such atool_choiceto{"type":"function","name":"litellm_code_execution"}, and leaves unrelated values like"auto"untouched. Covered bytest_pre_call_rewrites_forced_code_interpreter_tool_choiceandtest_pre_call_leaves_unrelated_tool_choice_untouched.Two more interceptor tests assert the output fed back to the model on a sandbox execution error (
[execution error] ...) and on unparseable tool arguments ([invalid tool arguments: could not parse code]), with no code run in the latter case.Update: sandbox isolation and registry hardening
The per-request sandbox cache is keyed on a server-minted random token rather than the caller-controlled
litellm_call_id(which the proxy sources from thex-litellm-call-idheader). Two concurrent requests that send a colliding call id can no longer share a sandbox container and read each other's code or files. The token is minted in the pre-call hook when interception activates, stripped from client requests at the proxy boundary alongside the other agentic-loop control markers, and survives the server-driven followups so a single request still reuses one sandbox across the loop.The proxy now registers sandbox tools unconditionally with an empty-list fallback, so a config reload that removes
sandbox_toolsclears the previously registered credentials instead of leaving them resolvable in the process. A client also cannot force the completed response to be re-wrapped as a synthetic stream it never requested, since_code_interpreter_interception_converted_streamis stripped at the request boundary too.Update: outputs parity and loop-limit hardening
The re-injected
code_interpreter_call.outputsis now an OpenAI-shaped logs array ([{"type": "logs", "logs": stdout}], or[]when there is no stdout) rather thannull, so a client that iterates overcode_interpreter_call.outputsor validates the response through the OpenAI SDK's Pydantic model gets the same shape as OpenAI's native container.max_agentic_loopsis also stripped at the proxy request boundary so a caller cannot raise the agentic-loop ceiling to drive many upstream model calls and sandbox executions from a single request; the loop stays bounded by the server default.