Repository navigation
feat(sandbox): code interpreter interceptor on the Responses API #30905
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
krrish-berri-2
merged 20 commits into
litellm_internal_staging
from
litellm_code_interpreter_interceptor
Jun 21, 2026
Merged
Changes from 2 commits
Commits
Show all changes
20 commits
Select commit
Hold shift + click to select a range
a83c4df
feat(sandbox): code interpreter interceptor on the Responses API
krrish-berri-2 775eaf8
feat(sandbox): re-inject code_interpreter_call so the response matche…
krrish-berri-2 7e8e304
feat(sandbox): support streaming for the code interpreter interceptor
krrish-berri-2 17f0098
fix(lint): use builtin generics in code interpreter interceptor to sa…
krrish-berri-2 b2648f3
fix(code-interpreter): gate sandbox execution, delete sandboxes, hard…
krrish-berri-2 f4e59f9
fix(lint): use PEP 604 X | None unions to satisfy UP045 budget
krrish-berri-2 26d9a10
test(code-interpreter): cover execution-error and unparseable-argumen…
krrish-berri-2 8b9a557
fix(code-interpreter): rewrite forced code_interpreter tool_choice to…
krrish-berri-2 9cdd9e6
test(sandbox): cover sandbox-tool registry resolution, reload clearin…
krrish-berri-2 d7e2864
test(code-interpreter): cover dict-shaped responses and object-attrib…
krrish-berri-2 722a976
fix(proxy): strip client-supplied _code_interpreter_interception_conv…
krrish-berri-2 ac87cf4
fix(code-interpreter): isolate sandboxes by server-minted key and cle…
krrish-berri-2 99908ca
refactor(sandbox): swap the tool registry atomically on reload
krrish-berri-2 935424f
fix(code-interpreter): cap caller loop limit and emit OpenAI-shaped o…
krrish-berri-2 61bda40
fix(sandbox): skip malformed sandbox_tools entries instead of crashin…
krrish-berri-2 002f067
chore: re-trigger CI and review on sandbox_tools hardening
krrish-berri-2 66594ad
chore: drop the code interpreter interception docs page from this PR
krrish-berri-2 1a79c09
fix(code-interpreter): delete sandbox in a finally when the agentic r…
krrish-berri-2 ecee105
fix(sandbox): skip tool entries missing sandbox_provider and keep UP0…
krrish-berri-2 c9ad727
fix(code-interpreter): delete cached sandbox when plan building fails
krrish-berri-2 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
74 changes: 74 additions & 0 deletions
74
docs/my-website/docs/proxy/code_interpreter_interception.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| # Code Interpreter Interception | ||
|
|
||
| Route the OpenAI `code_interpreter` tool to a self-hosted or e2b sandbox, transparently, without changing the request your client sends. | ||
|
|
||
| When a client calls the Responses API with a `code_interpreter` tool, LiteLLM swaps the native tool for a function tool, runs the python the model emits inside a sandbox you control, feeds the captured stdout back to the model through an agentic loop, and returns the final response. The client sends a normal `/v1/responses` request and gets a normal response back; the sandbox round-trips happen on the proxy. | ||
|
|
||
| ## How it works | ||
|
|
||
| 1. The pre-call hook detects a `code_interpreter` tool in the request and replaces it with a `litellm_code_execution` function tool, so the model emits code as function-call arguments. | ||
| 2. When the model returns a `litellm_code_execution` call, the interceptor executes the code in a sandbox (reused per request so state persists across iterations of the same call). | ||
| 3. The captured stdout is appended as `function_call_output` and the request is rerun, looping until the model produces a final answer. | ||
|
|
||
| ## Setup | ||
|
|
||
| ### 1. Define a sandbox tool and enable the callback | ||
|
|
||
| Register a sandbox under the top-level `sandbox_tools`, then turn on the interceptor with `callbacks` and point `code_interpreter_interception_params.sandbox_tool_name` at it. | ||
|
|
||
| ```yaml showLineNumbers title="config.yaml" | ||
| model_list: | ||
| - model_name: gpt-5 | ||
| litellm_params: | ||
| model: openai/gpt-5 | ||
| api_key: os.environ/OPENAI_API_KEY | ||
|
|
||
| sandbox_tools: | ||
| - sandbox_tool_name: my-e2b-sandbox | ||
| litellm_params: | ||
| sandbox_provider: e2b | ||
| api_key: os.environ/E2B_API_KEY | ||
| api_base: os.environ/E2B_API_BASE # Optional; set for self-hosted e2b | ||
|
|
||
| litellm_settings: | ||
| callbacks: ["code_interpreter_interception"] | ||
| code_interpreter_interception_params: | ||
| enabled_providers: ["openai"] | ||
| sandbox_tool_name: my-e2b-sandbox | ||
| ``` | ||
|
|
||
| `enabled_providers` scopes interception to specific upstream providers. Omit it to intercept for every provider. `sandbox_tool_name` must match a `sandbox_tool_name` registered under `sandbox_tools`. | ||
|
|
||
| ### 2. Set environment variables | ||
|
|
||
| ```shell | ||
| export OPENAI_API_KEY="sk-..." | ||
| export E2B_API_KEY="e2b_..." | ||
| ``` | ||
|
|
||
| ### 3. Start the proxy | ||
|
|
||
| ```shell | ||
| litellm --config config.yaml | ||
| ``` | ||
|
|
||
| ## Usage | ||
|
|
||
| Call `/v1/responses` with a `code_interpreter` tool. Nothing in the request is LiteLLM-specific; this is a standard OpenAI Responses API call. | ||
|
|
||
| ```shell | ||
| curl http://localhost:4000/v1/responses \ | ||
| -H "Content-Type: application/json" \ | ||
| -H "Authorization: Bearer $LITELLM_API_KEY" \ | ||
| -d '{ | ||
| "model": "gpt-5", | ||
| "input": "Compute the 20th Fibonacci number using python and tell me the result.", | ||
| "tools": [{"type": "code_interpreter", "container": {"type": "auto"}}] | ||
| }' | ||
| ``` | ||
|
|
||
| The model writes python, the proxy runs it in your sandbox, and the final response reflects the executed result. | ||
|
|
||
| ## Limitations (v0) | ||
|
|
||
| File upload and download are not supported yet. The interceptor executes code and returns stdout; passing input files into the sandbox or retrieving generated files out of it is not wired up. Only stdout from the executed code is fed back to the model. | ||
15 changes: 15 additions & 0 deletions
15
litellm/integrations/code_interpreter_interception/__init__.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| """ | ||
| Code Interpreter Interception Module | ||
|
|
||
| Converts the native OpenAI Responses ``code_interpreter`` tool into a function | ||
| tool, runs the model-emitted code in a sandbox, and feeds the result back into | ||
| the agentic loop. | ||
| """ | ||
|
|
||
| from litellm.integrations.code_interpreter_interception.handler import ( | ||
| CodeInterpreterInterceptionLogger, | ||
| ) | ||
|
|
||
| __all__ = [ | ||
| "CodeInterpreterInterceptionLogger", | ||
| ] |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.