fix(anthropic): return max_tokens stop_reason instead of 400 for tiny max_tokens on GPT-5.x

[devin-ai-integration]

TLDR

Problem this solves:

• Claude Code /model probe (max_tokens=1) 400s on GPT-5.x
• The Anthropic gateway marks the model unavailable

How it solves it:

• Map the provider output-limit 400 to stop_reason="max_tokens"
• Same for streaming and non-streaming, other errors still raise

User Flow

Before: a developer running Claude Code against LiteLLM as an Anthropic gateway cannot switch to a GPT-5.x model, because the client's availability probe comes back as an error

1. They start Claude Code with ANTHROPIC_BASE_URL=http://litellm-domain:4000 and run /model gpt-5.2
2. Claude Code probes the model by sending POST http://litellm-domain:4000/v1/messages with {"model":"gpt-5.2","max_tokens":1,"messages":[{"role":"user","content":"hello"}]}
3. The gateway answers 400 with litellm.BadRequestError: OpenAIException - Could not finish the message because max_tokens or model output limit was reached. Please try again with higher max_tokens.
4. Claude Code reads the 400 as "this model is unavailable" and refuses the switch
5. Sending the same request by hand with "max_tokens": 16 returns 200 with real content and "stop_reason": "end_turn", so the model and the key are healthy

After: the same probe comes back as an ordinary truncated turn, and the model switch goes through

1. They run /model gpt-5.2 against the same gateway
2. Claude Code sends the identical POST http://litellm-domain:4000/v1/messages with "max_tokens": 1
3. The gateway answers 200 with "stop_reason": "max_tokens", an empty text block, and zero output_tokens
4. Claude Code reads a successful response and allows the switch, then chats normally
5. The same probe with "stream": true returns a well-formed SSE lifecycle ending in "stop_reason": "max_tokens" instead of an error event
6. "max_tokens": 16 and above behave exactly as before, returning real content and "stop_reason": "end_turn"
7. An unrelated bad request, for example an out-of-range temperature, still comes back as a 400 with the provider's message

Relevant issues

Fixes #35061

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Live proxy hitting the real OpenAI API (GPT-5.x), no mocks. Config routes the Anthropic /v1/messages path through chat/completions (use_chat_completions_url_for_anthropic_messages: true), which is the path issue #35061 hits and the one used for Azure GPT-5.x and other OpenAI-compatible reasoning backends

Proxy started with

LITELLM_LOCAL_MODEL_COST_MAP=True python litellm/proxy/proxy_cli.py --config config.yaml --port 4000

Before the fix (base c274cf321c), the Claude Code probe returns 400 verbatim from the issue

$ curl -s -w "\nHTTP %{http_code}\n" http://localhost:4000/v1/messages \
    -H "content-type: application/json" -H "x-api-key: sk-1234" \
    -d '{"model":"gpt-5.2","max_tokens":1,"messages":[{"role":"user","content":"hello"}]}'
{"error":{"message":"litellm.BadRequestError: OpenAIException - Could not finish the message because max_tokens or model output limit was reached. Please try again with higher max_tokens.. Received Model Group=gpt-5.2\nAvailable Model Group Fallbacks=None","type":"invalid_request_error","param":null,"code":"400"}}
HTTP 400

After the fix (fbbebf5044), the same probe succeeds with stop_reason="max_tokens"

$ curl -s -w "\nHTTP %{http_code}\n" http://localhost:4000/v1/messages \
    -H "content-type: application/json" -H "x-api-key: sk-1234" \
    -d '{"model":"gpt-5.2","max_tokens":1,"messages":[{"role":"user","content":"hello"}]}'
{"id":"chatcmpl-f2856a1c-57c7-409d-8813-89747885710d","type":"message","role":"assistant","model":"gpt-5.2","stop_sequence":null,"usage":{"input_tokens":0,"output_tokens":0},"content":[{"type":"text","text":""}],"stop_reason":"max_tokens"}
HTTP 200

Streaming (fbbebf5044) emits a well-formed Anthropic SSE lifecycle ending in stop_reason: max_tokens

$ curl -s -N http://localhost:4000/v1/messages \
    -H "content-type: application/json" -H "x-api-key: sk-1234" \
    -d '{"model":"gpt-5.2","max_tokens":1,"stream":true,"messages":[{"role":"user","content":"hello"}]}'
event: message_start
data: {"type": "message_start", "message": {"id": "msg_5a21f61b-9a2c-4e10-b3e7-846b83d10910", "type": "message", "role": "assistant", "content": [], "model": "gpt-5.6", "stop_reason": null, "stop_sequence": null, "usage": {"input_tokens": 0, "output_tokens": 0, "cache_creation_input_tokens": 0, "cache_read_input_tokens": 0}}}

event: content_block_start
data: {"type": "content_block_start", "index": 0, "content_block": {"type": "text", "text": ""}}

event: content_block_stop
data: {"type": "content_block_stop", "index": 0}

event: message_delta
data: {"type": "message_delta", "delta": {"stop_reason": "max_tokens"}, "usage": {"input_tokens": 7, "output_tokens": 1}}

event: message_stop
data: {"type": "message_stop"}

Control (fbbebf5044): a normal budget is untouched and still finishes with real content and stop_reason="end_turn"

$ curl -s -w "\nHTTP %{http_code}\n" http://localhost:4000/v1/messages \
    -H "content-type: application/json" -H "x-api-key: sk-1234" \
    -d '{"model":"gpt-5.2","max_tokens":64,"messages":[{"role":"user","content":"hello"}]}'
{"id":"chatcmpl-E6t1fKOO6sGwyUNlRaIe5dmJ5LMJW","type":"message","role":"assistant","model":"gpt-5.2","stop_sequence":null,"usage":{"input_tokens":7,"output_tokens":10},"content":[{"type":"text","text":"Hello! How can I help?"}],"stop_reason":"end_turn"}
HTTP 200

The merge commit 75ee8d45dc that brought the branch up to current staging did not change any of this behavior; it only moved the guard into two helpers so the call sites keep staging's single-assignment completion_response: Final = ... shape. The regression suite and the 596 tests under tests/test_litellm/llms/anthropic/experimental_pass_through/ pass at that commit

Type

🐛 Bug Fix

Changes

Claude Code sends a max_tokens=1 probe when switching models with /model. On GPT-5.x that budget is too small for the model to finish even one token, so OpenAI rejects the request with a 400 ("Could not finish the message because max_tokens or model output limit was reached"). The Anthropic /v1/messages chat/completions translation path in LiteLLMMessagesToCompletionTransformationHandler surfaced that verbatim as a BadRequestError, so Claude Code concluded the model was unavailable. Anthropic itself answers the same probe with HTTP 200 and stop_reason="max_tokens", so the gateway should too

Both the async and sync handlers now go through a wrapper around the completion call that catches that specific provider 400 (identified by the output-limit marker in the message). When it matches, it synthesizes an empty finish_reason="length" ModelResponse and feeds it through the existing translation path, which already maps length to Anthropic max_tokens; for streaming it wraps that response in MockResponseIterator so the same SSE adapter produces the normal message lifecycle. Any other BadRequestError re-raises unchanged, so unrelated failures keep their current behavior

async def _acompletion_allowing_max_tokens_truncation(completion_kwargs, model, stream):
    try:
        return await litellm.acompletion(**completion_kwargs)
    except Exception as e:
        if not _is_output_token_limit_error(e):
            raise
        truncated = _build_max_tokens_truncation_response(model)  # empty, finish_reason="length"
        return MockResponseIterator(model_response=truncated) if stream else truncated

The sync helper is the same shape over litellm.completion, and each call site stays a single completion_response: Final = ... assignment

Regression coverage lives in tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_handler_max_tokens_truncation.py and asserts the async/sync non-streaming responses carry stop_reason="max_tokens", the streaming path emits the max_tokens SSE, and an unrelated BadRequestError still propagates

Caveats (if any)

• Overlaps fix(openai,azure): return a length-truncated 200 when the output budget fits no token #36859, which maps the same 400 in the OpenAI/Azure helpers
• If that lands, this only backstops other providers hitting the same error
• choices=[...] needs mutable-ok: ModelResponse.__init__ ignores non-list choices

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/a1b43c1b83a14a26801c3dcb35b11959

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

The PR narrowly translates provider output-limit errors in the Anthropic Messages-to-completions adapter into successful stop_reason="max_tokens" responses.

• Adds equivalent handling for synchronous and asynchronous completion calls.
• Uses the existing response and SSE translation paths for non-streaming and streaming requests.
• Adds regression coverage for translated output-limit errors and propagation of unrelated bad requests.

Confidence Score: 5/5

The PR appears safe to merge, with the targeted error translation preserving unrelated failures and using the established Anthropic response adapters.

The matched provider error is converted into the normalized length finish reason for both synchronous and asynchronous paths, while the streaming iterator supports the required protocols and unrelated bad requests continue to propagate.

Important Files Changed

Filename	Overview
litellm/llms/anthropic/experimental_pass_through/adapters/handler.py	Adds narrowly matched output-limit error translation into a synthetic length-finished response for sync, async, and streaming paths; no actionable defect was established.
tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_handler_max_tokens_truncation.py	Adds isolated mocked regression tests covering async and sync responses, streaming SSE behavior, and unrelated-error propagation.

_{Reviews (1): Last reviewed commit: "refactor(anthropic): drop getattr and tr..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 86.48649% with 5 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...opic/experimental_pass_through/adapters/handler.py	86.48%	5 Missing ⚠️

📢 Thoughts on this report? Let us know!

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_fix_35061_max_tokens_probe (75ee8d4) with litellm_internal_staging (8841cbc)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (2b63919) during the generation of this report, so 8841cbc was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[yassin-berriai]

Opened #36859, which widens this same mapping down to the OpenAI and Azure chat request helpers, so direct /v1/chat/completions callers and streaming are covered too.

[devin-ai-integration]

Yours is the more general fix and no files overlap. Want me to keep this as a backstop for other providers, or close it?

[tin-berri]

Closing as stale: no activity in the last 30 days. Feel free to reopen if this is still relevant.