test(responses): bound azure shell tool live call at 90s and skip on provider timeout

[mateo-berri]

Relevant issues

Related to #32420, which bounds the suite's daily cassette re-record calls at the job level and reruns timeout-class failures once. This PR covers the one test in the suite that goes live on every run and so needs its own bound regardless of cassette state

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

CircleCI job 2013288 (llm_responses_api_testing) hung for 15 minutes and was killed by Too long with no output (exceeded 15m0s) at 98%. The hung test was test_azure_responses_api.py::TestAzureResponsesAPITest::test_responses_api_shell_tool: it always runs live against Azure because its skip outcome ("shell not supported for this model") means the VCR persister never saves a cassette for it, and on that run Azure accepted the request and never answered. litellm's Responses API surface uses litellm.request_timeout = 6000s as its default HTTP deadline (timeout or request_timeout in litellm/responses/main.py), so no client-side timeout fired inside the 15m no-output window and the whole job died instead of the one test

A hang like this cannot be reproduced against live Azure on demand (the same request normally answers in well under a second; on staging job 2013697 the test skips in 0.334s), so the repro below points AZURE_AI_API_BASE at a real local TCP endpoint that accepts connections and never responds, which is exactly the observed provider behavior. No code is mocked; the test exercises the full litellm -> aiohttp stack

Black-hole endpoint used by both runs:

python3 -c "
import socket, threading, time
srv = socket.socket(); srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
srv.bind(('127.0.0.1', 9321)); srv.listen(16)
def hold(c):
    c.recv(65536); time.sleep(7200)
while True:
    c, _ = srv.accept(); threading.Thread(target=hold, args=(c,), daemon=True).start()
"

Before (4b0ac8b, the current base): the test hangs indefinitely, reproducing the CI incident; the run below produced no verdict and had to be killed externally after 150s

$ AZURE_AI_API_BASE=http://127.0.0.1:9321 AZURE_AI_API_KEY=dummy LITELLM_VCR_DISABLE=1 \
    perl -e 'alarm 150; exec @ARGV' python -m pytest \
    "tests/llm_responses_api_testing/test_azure_responses_api.py::TestAzureResponsesAPITest::test_responses_api_shell_tool" -v --no-header
============================= test session starts ==============================
collecting ... collected 1 item

tests/llm_responses_api_testing/test_azure_responses_api.py::TestAzureResponsesAPITest::test_responses_api_shell_tool
[killed by alarm after 150s, no verdict ever printed]

After (d3b5294): the same hang now resolves as a skip in 90s, the same graceful outcome the test already uses when Azure rejects the shell tool with a 400

$ AZURE_AI_API_BASE=http://127.0.0.1:9321 AZURE_AI_API_KEY=dummy LITELLM_VCR_DISABLE=1 \
    python -m pytest \
    "tests/llm_responses_api_testing/test_azure_responses_api.py::TestAzureResponsesAPITest::test_responses_api_shell_tool" -v --no-header -rs
============================= test session starts ==============================
collecting ... collected 1 item

tests/llm_responses_api_testing/test_azure_responses_api.py::TestAzureResponsesAPITest::test_responses_api_shell_tool SKIPPED [100%]

=========================== short test summary info ============================
SKIPPED [1] tests/llm_responses_api_testing/base_responses_api.py:771: Provider did not answer the shell tool request within 90s
======================== 1 skipped in 90.10s (0:01:30) =========================

Type

✅ Test

Changes

tests/llm_responses_api_testing/base_responses_api.py: test_responses_api_shell_tool now passes timeout=90 to litellm.aresponses and skips on litellm.Timeout, mirroring its existing skips for InternalServerError and shell-not-supported BadRequestError. This is the one test in the suite that goes live on every run (a skipping test never persists a cassette), so a provider-side hang previously inherited the Responses API default deadline of 6000s and ate the job's 15m no-output window

An earlier revision of this PR also added a job-level pytest timeout of 120s to the CircleCI job. That was dropped: #32420 bounds the same job's live calls at 180s via REQUEST_TIMEOUT and reruns timeout-class failures once, and a 120s per-test kill would fire before that 180s bound, terminating the worker process instead of surfacing a rerunnable litellm.Timeout. The explicit per-call timeout=90 here composes cleanly with #32420 because a per-request timeout takes precedence over the job-level default

[greptile-apps]

Greptile Summary

This PR adds a 90-second client-side timeout and a litellm.Timeout skip handler to test_responses_api_shell_tool in base_responses_api.py, preventing a provider-side silence from hanging the entire CI job as happened in CircleCI job 2013288.

• Adds timeout=90 to the litellm.aresponses call so the test self-limits rather than inheriting the Responses API default of 6000s.
• Adds an except litellm.Timeout branch that skips gracefully, matching the existing skip paths for InternalServerError and shell-not-supported BadRequestError.

Confidence Score: 5/5

Safe to merge — the change is confined to a test file, adds a hard timeout bound, and introduces a graceful skip path without weakening any assertions.

Only one file is touched: a single test method gains a 90-second request timeout and a skip handler for that timeout. No production code is changed, no assertions are removed, and the existing skip paths for InternalServerError and BadRequestError are preserved. The fix is well-motivated and directly addresses the documented CI hang.

No files require special attention.

Important Files Changed

Filename	Overview
tests/llm_responses_api_testing/base_responses_api.py	Adds timeout=90 and a Timeout skip handler to test_responses_api_shell_tool; no logic weakened, existing assertions untouched.

_{Reviews (2): Last reviewed commit: "test(responses): drop job-level pytest t..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 30 untouched benchmarks

---

_{Comparing litellm_bound_responses_shell_tool_e2e (d3b5294) with litellm_internal_staging (b2e2a38)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (7cc6608) during the generation of this report, so b2e2a38 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩