Skip to content

refactor(e2e/claude_code): dedupe tool_use cell body into shared _tool_use module - #33625

Open
mateo-berri wants to merge 1 commit into
litellm_internal_stagingfrom
litellm_claude_code_tool_use_dedupe
Open

refactor(e2e/claude_code): dedupe tool_use cell body into shared _tool_use module#33625
mateo-berri wants to merge 1 commit into
litellm_internal_stagingfrom
litellm_claude_code_tool_use_dedupe

Conversation

@mateo-berri

Copy link
Copy Markdown
Contributor

Relevant issues

Follow-up to #33474; Greptile flagged that _has_tool_use_event was copy-pasted across the tool_use cell files. Stacked on #33474 and must merge after it (base is that PR's branch)

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Live proxy run against real provider APIs (OpenAI, Azure OpenAI, Anthropic), captured at 62c00fb. The "before" reference run is the parent branch head a017b95, whose identical cells were proven live in #33474's runbook; behavior here is identical by construction (same prompt, CLI args, per-model checks, compat_result payloads, and failure strings, now sourced from one module)

Proxy start:

cd litellm && (set -a; source .env; set +a; uv run python litellm/proxy/proxy_cli.py --config tests/e2e/claude_code/test_config.yaml --port 53431)

Test collection is unchanged by the refactor (18 tests both before at a017b95 and after at 62c00fb):

$ uv run pytest tests/e2e/claude_code/tool_use tests/e2e/claude_code/tool_use_streaming --collect-only -q | tail -1
18 tests collected in 0.03s

Live cells through the shared body (openai and azure_openai each drive the three GPT-5.6 tiers; anthropic covers a legacy Claude column):

$ LITELLM_PROXY_URL=http://localhost:53431 LITELLM_MASTER_KEY=$LITELLM_MASTER_KEY uv run pytest \
    tests/e2e/claude_code/tool_use/test_openai.py \
    tests/e2e/claude_code/tool_use/test_azure_openai.py \
    tests/e2e/claude_code/tool_use/test_anthropic.py \
    tests/e2e/claude_code/tool_use_streaming/test_openai.py \
    tests/e2e/claude_code/tool_use_streaming/test_azure_openai.py \
    tests/e2e/claude_code/tool_use_streaming/test_anthropic.py -q
.E2E_RESULT package=claude_code file=test_openai.py outcome=passed duration_ms=7224 node_id=claude_code/tool_use/test_openai.py::test_tool_use_openai covers=""
.E2E_RESULT package=claude_code file=test_azure_openai.py outcome=passed duration_ms=7161 node_id=claude_code/tool_use/test_azure_openai.py::test_tool_use_azure_openai covers=""
.E2E_RESULT package=claude_code file=test_anthropic.py outcome=passed duration_ms=6487 node_id=claude_code/tool_use/test_anthropic.py::test_tool_use_anthropic covers=llm.messages.anthropic.tool_use.nonstream.works
.E2E_RESULT package=claude_code file=test_openai.py outcome=passed duration_ms=6193 node_id=claude_code/tool_use_streaming/test_openai.py::test_tool_use_streaming_openai covers=""
.E2E_RESULT package=claude_code file=test_azure_openai.py outcome=passed duration_ms=5447 node_id=claude_code/tool_use_streaming/test_azure_openai.py::test_tool_use_streaming_azure_openai covers=""
.E2E_RESULT package=claude_code file=test_anthropic.py outcome=passed duration_ms=6048 node_id=claude_code/tool_use_streaming/test_anthropic.py::test_tool_use_streaming_anthropic covers=llm.messages.anthropic.tool_use.stream.works
6 passed in 39.71s

Type

🧹 Refactoring
✅ Test

Changes

The 16 live cell files under tests/e2e/claude_code/tool_use/ and tests/e2e/claude_code/tool_use_streaming/ each carried an identical copy of _has_tool_use_event, the TOOL_USE_PROMPT/TOOL_USE_ARGS constants, and the per-model outcome-assertion loop (the streaming files additionally duplicated _count_input_json_deltas and one extra check). This PR extracts all of it into tests/e2e/claude_code/_tool_use.py, mirroring the existing _basic_messaging.py convention: each cell file keeps its module docstring, model list, and @pytest.mark.covers marker, and now calls the shared run_tool_use_cell(...) (with verify_streaming=True for the streaming feature). The Bash allowlist security rationale moved from a comment in tool_use/test_anthropic.py into the shared module's docstring, and the mantle cells keep their skip_unless_mantle_cells_enabled() gate in place

Behavior is identical by construction: same skip semantics, same compat_result.add payloads, same failure message formats, and the same pytest.fail(..., pytrace=False) shape; the loop bodies were verified byte-identical across all 16 files (modulo the models constant name) before extraction. Net diff is about 1,300 duplicated lines removed

QA runbook

Environment prerequisites are the same as #33474: a proxy booted from tests/e2e/claude_code/test_config.yaml, ANTHROPIC_API_KEY, OPENAI_API_KEY with chat quota, and AZURE_API_BASE/AZURE_API_KEY pointing at a resource with gpt-5.6-sol/-terra/-luna deployments; mantle cells stay opt-in via COMPAT_MANTLE_CELLS=1. The <alias> placeholders run once per model in the cell's list

  • tests/e2e/claude_code/tool_use/test_openai.py::test_tool_use_openai (same shape for every other tool_use/ cell; only the model list differs) - a tool call round-trips to an Anthropic tool_use block through the shared run_tool_use_cell body
    • ANTHROPIC_BASE_URL=http://localhost:53431 ANTHROPIC_AUTH_TOKEN=$LITELLM_MASTER_KEY claude --model <alias> -p "Use the Bash tool to run the command `echo pong` and report what it printed." --allowed-tools "Bash(echo pong)" --permission-mode dontAsk --output-format stream-json --verbose
    • Expect an assistant event whose content contains a tool_use block for each model in the cell's list
    • Expect a red cell to report the exact same failure strings as before this PR ([<model>] no tool_use content block observed in stream-json events, CLI error, or non-zero exit diagnostics)
    • Sanity check: this test makes sense to add and is not hand-wavey (e.g., assert actual expected spend instead of just spend > 0) or potentially flaky
  • tests/e2e/claude_code/tool_use_streaming/test_openai.py::test_tool_use_streaming_openai (same shape for every other tool_use_streaming/ cell; only the model list differs) - streamed tool arguments arrive as incremental input_json_delta events via the same shared body with verify_streaming=True
    • Run the tool_use command above with --include-partial-messages added
    • Expect a tool_use block plus at least one content_block_delta stream event whose delta type is input_json_delta; zero deltas means the proxy collapsed the streamed tool input
    • Sanity check: this test makes sense to add and is not hand-wavey (e.g., assert actual expected spend instead of just spend > 0) or potentially flaky

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

@greptile-apps

greptile-apps Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR extracts the duplicated _has_tool_use_event, _count_input_json_deltas, TOOL_USE_PROMPT/TOOL_USE_ARGS constants, and the per-model outcome-assertion loop from all 16 tool_use / tool_use_streaming cell files into a single shared module at tests/e2e/claude_code/_tool_use.py, mirroring the existing _basic_messaging.py convention.

  • Each cell file now declares only its model list and calls run_tool_use_cell(compat_result=..., models=...), with streaming cells additionally passing verify_streaming=True; the security rationale for the Bash allowlist is consolidated in the new module's docstring.
  • The refactoring is behavior-preserving: skip semantics, compat_result.add payloads, failure message formats, and pytest.fail(..., pytrace=False) shape are identical; TOOL_USE_ARGS changed from lists to tuples (concatenation-safe), and pytest imports were correctly dropped only from files that no longer reference pytest directly.

Confidence Score: 5/5

Safe to merge — pure mechanical extraction of identical code into a shared module with no behavioral changes and live test runs confirming all 18 tests pass.

Every cell's outcome-assertion loop, constants, and helper functions were byte-verified as identical before extraction. The shared module faithfully preserves the original skip semantics, per-model error messages, and streaming check. Imports were trimmed correctly: files with @pytest.mark.covers keep import pytest; those without do not. The only structural change (TOOL_USE_ARGS list to tuple) is compatible everywhere it is consumed. Live proxy runs across OpenAI, Azure OpenAI, and Anthropic confirm no regressions.

No files require special attention.

Important Files Changed

Filename Overview
tests/e2e/claude_code/_tool_use.py New shared module consolidating _has_tool_use_event, _count_input_json_deltas, constants, and run_tool_use_cell; logic is correct and well-documented
tests/e2e/claude_code/tool_use/test_anthropic.py Replaced inline duplicated body with run_tool_use_cell(...); @pytest.mark.covers marker and import pytest retained correctly
tests/e2e/claude_code/tool_use/test_azure.py Replaced inline body with run_tool_use_cell(...); security rationale pointer updated to new module location
tests/e2e/claude_code/tool_use/test_azure_openai.py Replaced inline body with run_tool_use_cell(...); pytest import correctly dropped since this cell has no @pytest.mark.covers decorator
tests/e2e/claude_code/tool_use/test_bedrock_converse.py Replaced inline body with run_tool_use_cell(...); @pytest.mark.covers marker and pytest import retained correctly
tests/e2e/claude_code/tool_use/test_bedrock_invoke.py Replaced inline body with run_tool_use_cell(...); @pytest.mark.covers marker and pytest import retained correctly
tests/e2e/claude_code/tool_use/test_bedrock_mantle.py Replaced inline body with run_tool_use_cell(...); skip_unless_mantle_cells_enabled() gate is preserved before the shared call
tests/e2e/claude_code/tool_use/test_openai.py Replaced inline body with run_tool_use_cell(...); pytest import correctly dropped since no @pytest.mark.covers decorator
tests/e2e/claude_code/tool_use/test_vertex_ai.py Replaced inline body with run_tool_use_cell(...); @pytest.mark.covers marker and pytest import retained correctly
tests/e2e/claude_code/tool_use_streaming/test_anthropic.py Replaced inline streaming body with run_tool_use_cell(..., verify_streaming=True); @pytest.mark.covers and pytest import retained correctly
tests/e2e/claude_code/tool_use_streaming/test_azure.py Replaced inline streaming body with run_tool_use_cell(..., verify_streaming=True); @pytest.mark.covers and pytest import retained
tests/e2e/claude_code/tool_use_streaming/test_azure_openai.py Replaced inline streaming body with run_tool_use_cell(..., verify_streaming=True); pytest import correctly removed since no @pytest.mark.covers decorator present
tests/e2e/claude_code/tool_use_streaming/test_bedrock_converse.py Replaced inline streaming body with run_tool_use_cell(..., verify_streaming=True); @pytest.mark.covers and pytest import retained
tests/e2e/claude_code/tool_use_streaming/test_bedrock_invoke.py Replaced inline streaming body with run_tool_use_cell(..., verify_streaming=True); @pytest.mark.covers and pytest import retained
tests/e2e/claude_code/tool_use_streaming/test_bedrock_mantle.py Replaced inline streaming body with run_tool_use_cell(..., verify_streaming=True); skip_unless_mantle_cells_enabled() gate preserved
tests/e2e/claude_code/tool_use_streaming/test_openai.py Replaced inline streaming body with run_tool_use_cell(..., verify_streaming=True); pytest import correctly dropped since no @pytest.mark.covers decorator
tests/e2e/claude_code/tool_use_streaming/test_vertex_ai.py Replaced inline streaming body with run_tool_use_cell(..., verify_streaming=True); @pytest.mark.covers and pytest import retained

Reviews (1): Last reviewed commit: "refactor(e2e/claude_code): dedupe tool_u..." | Re-trigger Greptile

@codecov

codecov Bot commented Jul 17, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Base automatically changed from litellm_claude_code_e2e_gpt_big3 to litellm_internal_staging July 17, 2026 01:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants