feat: add litellm.agent() to run claude code, codex, opencode and deep agents through the ai gateway

[ishaan-berri]

TLDR

Problem this solves:

• No way to run Claude Code, Codex, OpenCode or Deep Agents on any LiteLLM model
• Agent model calls skip the AI Gateway, so spend and keys are scattered

How it solves it:

• New litellm.agent() / litellm.aagent(), plus agent_session, agent_resume, agent_capabilities
• Harness is a typed enum; passing a string raises TypeError
• model="litellm_proxy/<group>" sends every agent model call through the AI Gateway
• Each harness is a config on BaseHarnessConfig (litellm/llms/<harness>/harness/transformation.py); one generic handler does the I/O
• The runtime only sees a per-session token; real keys stay on the host
• No new dependencies in litellm

What you get

import litellm
from litellm import Harness, sandbox

result = litellm.agent(
    Harness.CODEX,                        # or CLAUDE_CODE, OPENCODE, DEEPAGENTS
    "Create hello.py that prints 'hi from codex', then run it with python3.",
    sandbox=sandbox.local("./repo"),
    model="litellm_proxy/bedrock_mantle/openai.gpt-5.4",   # a model group on your AI Gateway
)

result.text         # 'Done — created `hello.py` and ran it successfully. ...'
result.files        # [FileChange(path='hello.py', kind='created', diff='--- a/hello.py ...')]
result.cost         # 0.05512  (USD, summed over every model call the agent made)
result.usage        # Usage(input_tokens=..., output_tokens=..., calls=2)
result.stop_reason  # 'done'

With stream=True you get typed events as they happen. Real output from all four harnesses on the same task, each routed through the gateway (metadata={"run": "pr-proof"} shows up in spend logs, tagged harness,<name>):

>>> litellm.agent(Harness.CLAUDE_CODE, ..., model='litellm_proxy/claude-haiku-4-5-20251001', stream=True)
  Text       "I'll create the hello.py file and run it."
  ToolCall   write  <tmp>/repo/hello.py
  ToolCall   bash   python3 <tmp>/repo/hello.py
  Text       'Done! Created hello.py and ran it successfully — it printed "hi from claude_code".'
  FileChange created hello.py
  Done       stop_reason=done calls=3 tokens=487 cost=$0.00233
  hello.py:  "print('hi from claude_code')"
>>> litellm.agent(Harness.CODEX, ..., model='litellm_proxy/bedrock_mantle/openai.gpt-5.4', stream=True)
  Text       'I’ll add the tiny script, then execute it to verify.'
  ToolCall   bash   /bin/zsh -lc "printf \"%s\n\" \"print('hi from codex')\" > hello.py && 
  Text       'Done — created `hello.py` and ran it successfully. - File: `hello.py:1` - Output: `hi from codex`'
  FileChange created hello.py
  Done       stop_reason=done calls=2 tokens=19293 cost=$0.05512
  hello.py:  "print('hi from codex')"
>>> litellm.agent(Harness.OPENCODE, ..., model='litellm_proxy/claude-haiku-4-5-20251001', stream=True)
  Text       "I'll create the hello.py file and then run it with python3."
  ToolCall   write  
  Text       'Now let me run it with python3:'
  ToolCall   bash   python3 hello.py
  Text       "Done! The hello.py file has been created and executed successfully. It prints 'hi from opencode' as expected."
  FileChange created hello.py
  Done       stop_reason=done calls=3 tokens=33688 cost=$0.03474
  hello.py:  "print('hi from opencode')"
>>> litellm.agent(Harness.DEEPAGENTS, ..., model='litellm_proxy/claude-haiku-4-5-20251001', stream=True)
  Text       "I'll create the hello.py file and then run it with python3."
  ToolCall   write  /hello.py
  ToolCall   bash   python3 /hello.py
  Text       'Let me check the current working directory and create the file there:'
  ToolCall   bash   pwd
  ToolCall   write  ./hello.py
  Text       'Now let me run it:'
  ToolCall   bash   python3 <tmp>/repo/hello.py
  Text       "Perfect! I've successfully created `hello.py` and run it. The output is: ``` hi from deepagents ``` The script"
  FileChange created hello.py
  Done       stop_reason=done calls=4 tokens=16204 cost=$0.01527
  hello.py:  "print('hi from deepagents')"

Sessions keep the runtime's own conversation across turns and can be saved and resumed in another process:

with litellm.agent_session(Harness.CLAUDE_CODE, sandbox=box, model="litellm_proxy/claude") as s:
    s.run("Remember the code word PELICAN.")
    s.run("What was the code word?").text   # 'PELICAN'
    state = s.detach().dumps()              # bytes; no credentials inside

litellm.agent_resume(State.loads(state), sandbox=box, model="litellm_proxy/claude")

User Flow

Before: a developer who wants Codex to fix a repo on their team's Bedrock models has no LiteLLM path

1. They install Codex and hand-edit ~/.codex/config.toml to point it at their LiteLLM gateway
2. They paste their virtual key into the runtime's env, where any tool call can read it
3. Spend shows up untagged on the gateway, mixed with every other request

After: the same developer runs Codex from Python through the gateway with one call

1. They pip install litellm starlette uvicorn, have codex on PATH, and set LITELLM_PROXY_API_BASE / LITELLM_PROXY_API_KEY
2. They call litellm.agent(Harness.CODEX, "fix the flaky test", sandbox=sandbox.local("./repo"), model="litellm_proxy/coder")
3. Codex edits the repo; the call returns the final text, the changed files and result.cost
4. The gateway spend logs show each request tagged harness,codex

Pre-Submission checklist

• I have added meaningful tests
• The handful of test files covering my change pass locally
• My PR passes all required CI/CD checks
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5

Screenshots / Proof of Fix

Real runs against https://gateway.litellm-sandbox.ai, real model calls, no mocks, real claude, codex and opencode CLIs and real deepagents.

Before (82d8b37)

All harnesses

1. python -c "import litellm; litellm.agent"
2. AttributeError: module 'litellm' has no attribute 'agent'

After (f76c2ac)

One task on each harness

1. python .claude/harness-docs/demo.py (the script behind the output under "What you get")
2. All four created and ran hello.py: Claude Code $0.00233, Codex $0.05512, OpenCode $0.03474, Deep Agents $0.01527

E2E suite, all four harnesses

1. LITELLM_PROXY_API_BASE=... LITELLM_PROXY_API_KEY=... pytest tests/harness_e2e -v
2. 26 passed. For each harness: creates a file and reports cost, streams events with Done last, returns validated structured output, remembers an earlier turn, detaches and resumes, read-only blocks writes. Plus a string harness is rejected and an unsupported option fails before the runtime starts

Unit tests

1. pytest tests/unit/harness tests/unit/llms/{base_llm/harness,claude_code,codex,opencode,deepagents}
2. 285 passed, 1 skipped (the skip is a live test that needs gateway env)

Type

🆕 New Feature
✅ Test

Caveats (if any)

Medium

• Runtimes aren't auto-installed; claude/codex/opencode must be on PATH
• permissions="ask" (interactive approval) only on Deep Agents for now
• CLI harnesses keep session transcripts in ~/.cache/litellm-harness so resume works
• Docker sandbox: local endpoint binds 127.0.0.1, not yet verified from inside containers
• tests/harness_e2e is allowlisted out of CI (needs live CLIs and a gateway); run it manually

Low

• OSV scan fails on urllib3/pyjwt/next lockfile versions; also red on main since 18:57 today, not from this PR
• Deep Agents resume across processes needs a persistent LangGraph checkpointer (in-process works)

Docs: BerriAI/litellm-docs#1945

[ishaan-berri]

@Veria please re-review at 8e8f811. Both open issues (OpenCode repo plugins, Deep Agents symlink escape) are fixed with regression tests; details in the earlier comment. @greptileai please re-review the latest commits.

[greptile-apps]

Custom HTTP handler violates requirement

This endpoint registers its own Starlette routes and uses its own handler to parse and dispatch model requests. The repository requires new HTTP handlers to reuse LiteLLM’s existing HTTP-handler infrastructure instead of implementing a custom handler.

Rule Used: What: Block any PR which writes it's own custom http handler instead of reusing litellm's existing http handlers. (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[ishaan-berri]

@greptileai thanks, addressed in d31d713:

1. Successful turns can lose events: fixed. _put_end now waits for room in the queue, so the end marker lands behind every event and nothing is dropped. It only discards queued events when the turn was cancelled, because then there is no consumer left to drain the queue. New test test_turn_keeps_every_event_when_queue_overflows pushes 40 events through a 4-slot queue and asserts all 40 arrive; it fails on the old code.
2. Nested file writes fail: fixed. The symlink check now resolves the nearest existing ancestor, so new_dir/sub/file.py is allowed. A path under a symlink that leaves the workspace is still refused. Tests: test_write_into_new_nested_directory_is_allowed, test_write_under_symlinked_dir_is_refused.
3. Custom HTTP handler: this endpoint is not an outbound HTTP handler, so there is no existing handler to reuse for it. It is a small local server the CLI runtimes (Claude Code, Codex, OpenCode) call as their model base URL. They only speak HTTP, so something has to listen on a port for them. Its outbound calls already go through LiteLLM's shared HTTP infrastructure: gateway mode uses the cached client from get_async_httpx_client, and SDK mode calls litellm.anthropic.messages.acreate / litellm.acompletion / litellm.aresponses. Please re-review with that in mind.

Please re-review.

[ishaan-berri]

@Veria please re-review at d31d713. Both of your open issues are fixed with regression tests (OpenCode runs with --pure; Deep Agents file tools refuse paths that resolve outside the workspace through symlinks).