fix(anthropic): honor messages request timeout

[MelvinOrichiSocana-hs]

Relevant issues

Fixes #26752

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

N/A

Screenshots / Proof of Fix

I verified this with a real LiteLLM proxy request to /v1/messages and a local slow Anthropic-compatible HTTP upstream. The local upstream sleeps for 1.25s, while the model config sets request_timeout: 0.3. This keeps the proof deterministic without sending a real provider request

Config used for both runs:

model_list:
  - model_name: claude-timeout-proof
    litellm_params:
      model: anthropic/claude-timeout-proof
      api_key: sk-ant-local-proof
      api_base: http://127.0.0.1:8999
      request_timeout: 0.3

general_settings:
  master_key: sk-1234

litellm_settings:
  drop_params: true
  modify_params: true
  num_retries: 0

Before fix, at base commit bf02a4a47f, the request waited for the slow upstream and returned 200, showing that the model-level timeout was not applied to the Anthropic /v1/messages HTTP request:

uv run --extra proxy python litellm/proxy/proxy_cli.py --config /private/tmp/litellm_timeout_proof_config.yaml --port 4010 --detailed_debug

curl -sS -w '\nhttp_code=%{http_code}\ntime_total=%{time_total}\n' \
  -H 'Authorization: Bearer sk-1234' \
  -H 'Content-Type: application/json' \
  -H 'anthropic-version: 2023-06-01' \
  --data '{"model":"claude-timeout-proof","max_tokens":1,"messages":[{"role":"user","content":"hi"}]}' \
  http://127.0.0.1:4010/v1/messages

{"id":"msg_timeout_proof","type":"message","role":"assistant","model":"claude-timeout-proof","content":[{"type":"text","text":"slow upstream response"}],"stop_reason":"end_turn","stop_sequence":null,"usage":{"input_tokens":5,"output_tokens":3}}
http_code=200
time_total=1.946157

After fix, at commit 63019add86, the same request returns a LiteLLM timeout. The response body shows the configured 0.3s timeout reached the HTTP request path:

uv run --extra proxy python litellm/proxy/proxy_cli.py --config /private/tmp/litellm_timeout_proof_config.yaml --port 4010 --detailed_debug

curl -sS -w '\nhttp_code=%{http_code}\ntime_total=%{time_total}\n' \
  -H 'Authorization: Bearer sk-1234' \
  -H 'Content-Type: application/json' \
  -H 'anthropic-version: 2023-06-01' \
  --data '{"model":"claude-timeout-proof","max_tokens":1,"messages":[{"role":"user","content":"hi"}]}' \
  http://127.0.0.1:4010/v1/messages

{"error":{"message":"litellm.Timeout: Connection timed out. Timeout passed=0.3, time taken=0.302 seconds. Received Model Group=claude-timeout-proof\nAvailable Model Group Fallbacks=None","type":"None","param":"None","code":"408"}}
http_code=408
time_total=1.512531

Local regression checks at commit 63019add86:

uv run --extra proxy pytest tests/test_litellm/llms/custom_httpx/test_llm_http_handler.py -q
make pre-commit

Both passed locally. The targeted regression test also fails if the handler-level timeout= handoff is removed, with assert None == 0.3

Type

Bug Fix

Changes

Anthropic /v1/messages now resolves per-request and configured timeouts through the shared completion timeout resolver, then forwards the resolved value into the async HTTP POST helper

Regression coverage checks timeout precedence and the full async /v1/messages handler wiring into the HTTP client

[CLAassistant]

All committers have signed the CLA.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing MelvinOrichiSocana-hs:feat/anthropic-messages-timeout (63019ad) with litellm_internal_staging (6d17f9e)}

[greptile-apps]

Greptile Summary

The /v1/messages Anthropic pass-through path was ignoring per-request and model-level timeouts because _async_post_anthropic_messages_with_http_error_retry never received a timeout value. This PR adds a _resolve_anthropic_messages_timeout static method that feeds stream_timeout/timeout/request_timeout/global-timeout through the shared CompletionTimeout.resolve pipeline, then passes the result into the HTTP post call.

• Introduces _resolve_anthropic_messages_timeout to resolve timeouts using CompletionTimeout.resolve, honoring stream_timeout, timeout, request_timeout, and the global litellm.request_timeout in priority order.
• Adds a timeout parameter to _async_post_anthropic_messages_with_http_error_retry (defaults to None for safe backwards compatibility) and wires it from async_anthropic_messages_handler.
• New tests cover four timeout-precedence cases and full handler-level forwarding; the updated FakeAsyncClient.post signature is a required compatibility fix, not a test weakening.

Confidence Score: 4/5

The change is narrowly scoped to the async Anthropic messages handler, reuses existing timeout-resolution infrastructure, and defaults to None (preserving prior behavior) when the new timeout argument is omitted.

The fix is correct and well-tested. A missing test case for stream=True without stream_timeout set and no combined timeout + request_timeout precedence test leave a small gap in explicit coverage, though the logic handles both correctly.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/llms/custom_httpx/llm_http_handler.py	Adds _resolve_anthropic_messages_timeout static method and threads the resolved timeout through _async_post_anthropic_messages_with_http_error_retry; logic is correct and uses the existing CompletionTimeout.resolve infrastructure.
tests/test_litellm/llms/custom_httpx/test_llm_http_handler.py	Adds two new mock-only tests covering timeout resolution precedence and end-to-end handler wiring; updates FakeAsyncClient.post signature to accept the new timeout kwarg, which is a necessary compatibility change.

_{Reviews (1): Last reviewed commit: "fix(anthropic): honor messages request t..." | Re-trigger Greptile}

[MelvinOrichiSocana-hs]

@shivamrawat1 Thank you for taking a look

[yucheng-berri]

merged in #33418