Skip to content

fix(proxy): stop shipping the literal string "None" as error type and param - #39536

Merged
mateo-berri merged 7 commits into
litellm_internal_stagingfrom
litellm_openai_error_payload_non_llm_routes
Sep 8, 2026
Merged

mateo-berri merged 7 commits into
litellm_internal_stagingfrom
litellm_openai_error_payload_non_llm_routes

Conversation

@mateo-berri

@mateo-berri mateo-berri commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • All five /v1/files routes answer "type": "None" and "param": "None"
  • Rerank, images, realtime, anthropic, and pass-through routes do the same
  • Those are strings, not OpenAI error types, so no SDK case matches them
  • param is typed nullable, so "None" is plain wrong there

How it solves it:

User Flow

Before: a developer whose app classifies gateway errors by error.type gets the string None on every /v1/files failure, so each one lands in their catch-all branch

  1. They upload a batch file with POST https://litellm-domain/v1/files (purpose=batch, target_model_names=gpt-5-mini) and get HTTP 200 with a long gateway file id
  2. They delete it with DELETE https://litellm-domain/v1/files/{file_id} and get HTTP 200 with the file object
  3. A later GET https://litellm-domain/v1/files/{file_id} (or a retried delete, or GET .../content) answers HTTP 404 {"error":{"message":"File not found: <file_id>","type":"None","param":"None","code":"404"}}
  4. Their handler switches on error.type, sees the string None, matches no OpenAI type, and reports "unknown error" instead of "that file is gone"
  5. It then reads error.param to name the offending field and renders "problem with field None", because a string arrived where JSON null was expected
  6. GET https://litellm-domain/v1/files?target_model_names=gpt-5-mini,gpt-5-nano and an upload with only expires_after[anchor] answer HTTP 400 with the same two strings
  7. Every failure on POST https://litellm-domain/v1/rerank, POST https://litellm-domain/v1/images/generations, POST https://litellm-domain/v1/realtime/client_secrets, and a configured pass-through route carries the same two strings
  8. A POST https://litellm-domain/v1/images/generations that times out upstream answers HTTP 408 with "type": null, while the same timeout on POST https://litellm-domain/v1/chat/completions answers invalid_request_error, so the same failure classifies differently per route
  9. A POST https://litellm-domain/v1/realtime/transcription_sessions for a model the gateway does not serve, a WebRTC offer to POST https://litellm-domain/v1/realtime/calls that the provider rejects, and a configured adapter route all answer the same two strings

After: the same failures carry a real OpenAI error type and a JSON null param, so the handler they already wrote classifies them

  1. They upload a batch file with POST https://litellm-domain/v1/files (purpose=batch, target_model_names=gpt-5-mini) and get HTTP 200 with a long gateway file id
  2. They delete it with DELETE https://litellm-domain/v1/files/{file_id} and get HTTP 200 with the file object
  3. A later GET https://litellm-domain/v1/files/{file_id} (or a retried delete, or GET .../content) answers HTTP 404 {"error":{"message":"File not found: <file_id>","type":"invalid_request_error","param":null,"code":"404"}}
  4. Their handler matches invalid_request_error and tells the caller the file is gone
  5. It reads error.param, gets null, and correctly reports that no single field was named
  6. GET https://litellm-domain/v1/files?target_model_names=gpt-5-mini,gpt-5-nano and an upload with only expires_after[anchor] answer HTTP 400 with invalid_request_error and null
  7. POST https://litellm-domain/v1/rerank, POST https://litellm-domain/v1/images/generations, and POST https://litellm-domain/v1/realtime/client_secrets carry invalid_request_error on a 400, the pass-through route carries internal_server_error on its 500, all with param as null
  8. Both timeouts answer HTTP 408 with "type": "invalid_request_error" and "param": null, the same shape on both routes
  9. The transcription session, the rejected WebRTC offer, and the adapter route answer invalid_request_error with param as null

Relevant issues

Fixes #40135

Linear ticket

Resolves LIT-7129

Part of LIT-6829

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Both legs run the same config against the same Postgres, each as one proxy with 2 uvicorn workers on its own random port, and differ only in the commit the proxy was booted from: Before is the merge base with litellm_internal_staging (82e6b84f5a), After is this PR's tip (b3bcd715e0). The file uploads and deletes hit the real OpenAI files API, and the live-traffic case is a real Anthropic completion, so both proxies were serving real provider traffic, not just error paths

The /v1/messages case answers the Anthropic envelope on both sides (staging now shapes it from the status code), so it is a regression check rather than a fix case. Same for the credentials case: only param changes there, the shared handler's type is #39555's job

Shared config (lit7129_qa_config_r3.yaml), batch payload (batch.jsonl), and boot command:

model_list:
  - model_name: gpt-5-mini
    litellm_params:
      model: openai/gpt-5-mini
      api_key: os.environ/OPENAI_API_KEY
  - model_name: claude-sonnet-5
    litellm_params:
      model: anthropic/claude-sonnet-5
      api_key: os.environ/ANTHROPIC_API_KEY
  - model_name: gpt-realtime-2.1-mini
    litellm_params:
      model: openai/gpt-realtime-2.1-mini
      api_key: os.environ/OPENAI_API_KEY

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  pass_through_endpoints:
    - path: "/unreachable-upstream"
      target: "http://127.0.0.1:1/nope"
      headers:
        content-type: application/json
    - path: "/anthropic-adapter/v1/messages"
      target: litellm.llms.anthropic.experimental_pass_through.adapters.handler.ANTHROPIC_ADAPTER

litellm_settings:
  drop_params: true
{"custom_id": "req-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gpt-5-mini", "messages": [{"role": "user", "content": "say hi"}]}}
export DATABASE_URL="postgresql://<user>:<password>@127.0.0.1:5432/litellm"
python litellm/proxy/proxy_cli.py --config lit7129_qa_config_r3.yaml --port "$PORT" --num_workers 2 --use_v2_migration_resolver

Before (82e6b84)

live traffic (claude-sonnet-5)

  1. curl -sS -X POST http://127.0.0.1:$PORT/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"claude-sonnet-5","messages":[{"role":"user","content":"Say ok"}]}' | jq "{model, content: .choices[0].message.content, usage}"
  2.  {
       "model": "claude-sonnet-5",
       "content": "Ok",
       "usage": {
         "completion_tokens": 26,
         "prompt_tokens": 10,
         "total_tokens": 36,
         "completion_tokens_details": {
           "reasoning_tokens": 20,
           "text_tokens": 6
         },
         "prompt_tokens_details": {
           "cached_tokens": 0,
           "text_tokens": 10,
           "cache_write_tokens": 0,
           "cache_creation_tokens": 0,
           "cache_creation_token_details": {
             "ephemeral_5m_input_tokens": 0,
             "ephemeral_1h_input_tokens": 0
           }
         },
         "cache_creation_input_tokens": 0,
         "cache_read_input_tokens": 0,
         "inference_geo": "global",
         "service_tier": "standard"
       }
     }

files: list and upload validation

  1. curl -sS -w "\nHTTP %{http_code}\n" "http://127.0.0.1:$PORT/v1/files?target_model_names=gpt-5-mini,gpt-5-nano" -H "Authorization: Bearer $LITELLM_MASTER_KEY"
  2.  {"error":{"message":"target_model_names on list files must be a list of one model name. Example: ['gpt-4o']","type":"None","param":"None","code":"400"}}
     HTTP 400
    
  3. curl -sS -w "\nHTTP %{http_code}\n" http://127.0.0.1:$PORT/v1/files -H "Authorization: Bearer $LITELLM_MASTER_KEY" -F purpose=batch -F target_model_names=gpt-5-mini -F "expires_after[anchor]=created_at" -F file=@batch.jsonl
  4.  {"error":{"message":"{'error': 'Both expires_after[anchor] and expires_after[seconds] must be provided if expires_after is specified'}","type":"None","param":"None","code":"400"}}
     HTTP 400
    

files: deleted file (get, delete, content)

  1. FILE_ID=$(curl -sS http://127.0.0.1:$PORT/v1/files -H "Authorization: Bearer $LITELLM_MASTER_KEY" -F purpose=batch -F target_model_names=gpt-5-mini -F file=@batch.jsonl | jq -r .id); echo "$FILE_ID"
  2.  bGl0ZWxsbV9wcm94eTphcHBsaWNhdGlvbi9vY3RldC1zdHJlYW07dW5pZmllZF9pZCw0NWE4YTg1OS1mMmEwLTQyOWYtYThkYS1kZDllMmFmODU0NTU7dGFyZ2V0X21vZGVsX25hbWVzLGdwdC01LW1pbmk7bGxtX291dHB1dF9maWxlX2lkLGZpbGUtUlVpVW9SOXdpVnI3TGlOQ0hYZGVMYTtsbG1fb3V0cHV0X2ZpbGVfbW9kZWxfaWQsZTBhZGMxZDg3NWQwNjIwMTA1MjEzYjAwYWI1OWYxNTQyYzg0OWU4ZTNlMWZjNDQ4MDljYzAzZjMyZmIxYzkyYw
    
  3. curl -sS -w "\nHTTP %{http_code}\n" -X DELETE "http://127.0.0.1:$PORT/v1/files/$FILE_ID" -H "Authorization: Bearer $LITELLM_MASTER_KEY"
  4.  {"id":"bGl0ZWxsbV9wcm94eTphcHBsaWNhdGlvbi9vY3RldC1zdHJlYW07dW5pZmllZF9pZCw0NWE4YTg1OS1mMmEwLTQyOWYtYThkYS1kZDllMmFmODU0NTU7dGFyZ2V0X21vZGVsX25hbWVzLGdwdC01LW1pbmk7bGxtX291dHB1dF9maWxlX2lkLGZpbGUtUlVpVW9SOXdpVnI3TGlOQ0hYZGVMYTtsbG1fb3V0cHV0X2ZpbGVfbW9kZWxfaWQsZTBhZGMxZDg3NWQwNjIwMTA1MjEzYjAwYWI1OWYxNTQyYzg0OWU4ZTNlMWZjNDQ4MDljYzAzZjMyZmIxYzkyYw","bytes":157,"object":"file","status":"uploaded","purpose":"batch","filename":"modified_file.jsonl","created_at":1788898403,"expires_at":1791490403,"status_details":null}
     HTTP 200
    
  5. curl -sS -w "\nHTTP %{http_code}\n" "http://127.0.0.1:$PORT/v1/files/$FILE_ID" -H "Authorization: Bearer $LITELLM_MASTER_KEY"
  6.  {"error":{"message":"File not found: bGl0ZWxsbV9wcm94eTphcHBsaWNhdGlvbi9vY3RldC1zdHJlYW07dW5pZmllZF9pZCw0NWE4YTg1OS1mMmEwLTQyOWYtYThkYS1kZDllMmFmODU0NTU7dGFyZ2V0X21vZGVsX25hbWVzLGdwdC01LW1pbmk7bGxtX291dHB1dF9maWxlX2lkLGZpbGUtUlVpVW9SOXdpVnI3TGlOQ0hYZGVMYTtsbG1fb3V0cHV0X2ZpbGVfbW9kZWxfaWQsZTBhZGMxZDg3NWQwNjIwMTA1MjEzYjAwYWI1OWYxNTQyYzg0OWU4ZTNlMWZjNDQ4MDljYzAzZjMyZmIxYzkyYw","type":"None","param":"None","code":"404"}}
     HTTP 404
    
  7. curl -sS -w "\nHTTP %{http_code}\n" -X DELETE "http://127.0.0.1:$PORT/v1/files/$FILE_ID" -H "Authorization: Bearer $LITELLM_MASTER_KEY"
  8.  {"error":{"message":"File not found: bGl0ZWxsbV9wcm94eTphcHBsaWNhdGlvbi9vY3RldC1zdHJlYW07dW5pZmllZF9pZCw0NWE4YTg1OS1mMmEwLTQyOWYtYThkYS1kZDllMmFmODU0NTU7dGFyZ2V0X21vZGVsX25hbWVzLGdwdC01LW1pbmk7bGxtX291dHB1dF9maWxlX2lkLGZpbGUtUlVpVW9SOXdpVnI3TGlOQ0hYZGVMYTtsbG1fb3V0cHV0X2ZpbGVfbW9kZWxfaWQsZTBhZGMxZDg3NWQwNjIwMTA1MjEzYjAwYWI1OWYxNTQyYzg0OWU4ZTNlMWZjNDQ4MDljYzAzZjMyZmIxYzkyYw","type":"None","param":"None","code":"404"}}
     HTTP 404
    
  9. curl -sS -w "\nHTTP %{http_code}\n" "http://127.0.0.1:$PORT/v1/files/$FILE_ID/content" -H "Authorization: Bearer $LITELLM_MASTER_KEY"
  10. {"error":{"message":"File not found: bGl0ZWxsbV9wcm94eTphcHBsaWNhdGlvbi9vY3RldC1zdHJlYW07dW5pZmllZF9pZCw0NWE4YTg1OS1mMmEwLTQyOWYtYThkYS1kZDllMmFmODU0NTU7dGFyZ2V0X21vZGVsX25hbWVzLGdwdC01LW1pbmk7bGxtX291dHB1dF9maWxlX2lkLGZpbGUtUlVpVW9SOXdpVnI3TGlOQ0hYZGVMYTtsbG1fb3V0cHV0X2ZpbGVfbW9kZWxfaWQsZTBhZGMxZDg3NWQwNjIwMTA1MjEzYjAwYWI1OWYxNTQyYzg0OWU4ZTNlMWZjNDQ4MDljYzAzZjMyZmIxYzkyYw","type":"None","param":"None","code":"404"}}
    HTTP 404
    

files: unknown provider id

  1. curl -sS -w "\nHTTP %{http_code}\n" http://127.0.0.1:$PORT/v1/files/file-doesnotexist -H "Authorization: Bearer $LITELLM_MASTER_KEY"
  2.  {"error":{"message":"Error code: 404 - {'error': {'message': 'No such File object: file-doesnotexist', 'type': 'invalid_request_error', 'param': 'id', 'code': None}, 'detail': {'message': 'No such File object: file-doesnotexist', 'code': None}}","type":"invalid_request_error","param":"id","code":"404"}}
     HTTP 404
    

rerank

  1. curl -sS -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:$PORT/v1/rerank -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"no-such-rerank-model","query":"hi","documents":["a","b"]}'
  2.  {"error":{"message":"400: {'error': '/rerank: Invalid model name passed in model=no-such-rerank-model. Call `/v1/models` to view available models for your key.'}","type":"None","param":"None","code":"400"}}
     HTTP 400
    

images

  1. curl -sS -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:$PORT/v1/images/generations -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"no-such-image-model","prompt":"a cat"}'
  2.  {"error":{"message":"400: {'error': '/image/generations: Invalid model name passed in model=no-such-image-model. Call `/v1/models` to view available models for your key.'}","type":"None","param":"None","code":"400"}}
     HTTP 400
    

realtime

  1. curl -sS -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:$PORT/v1/realtime/client_secrets -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"session":{"type":"realtime","model":"no-such-realtime"}}'
  2.  {"error":{"message":"400: {'error': '/realtime/client_secrets: Invalid model name passed in model=no-such-realtime. Call `/v1/models` to view available models for your key.'}","type":"None","param":"None","code":"400"}}
     HTTP 400
    

anthropic /v1/messages

  1. curl -sS -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:$PORT/v1/messages -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"gpt-5-mini","max_tokens":16,"messages":"not-a-list"}'
  2.  {"type":"error","error":{"type":"api_error","message":"string indices must be integers, not 'str'"}}
     HTTP 500
    

pass-through

  1. curl -sS -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:$PORT/unreachable-upstream -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"hi":"there"}'
  2.  {"error":{"message":"Cannot connect to host 127.0.0.1:1 ssl:<ssl.SSLContext object at 0x10cf15f50> [Connect call failed ('127.0.0.1', 1)]","type":"None","param":"None","code":"500"}}
     HTTP 500
    

shared handler (credentials)

  1. curl -sS -w "\nHTTP %{http_code}\n" http://127.0.0.1:$PORT/credentials/by_name/nope -H "Authorization: Bearer $LITELLM_MASTER_KEY"
  2.  {"error":{"message":"Credential not found. Got credential name: nope","type":"internal_server_error","param":"None","code":"404"}}
     HTTP 404
    

After (b3bcd71)

live traffic (claude-sonnet-5)

  1. curl -sS -X POST http://127.0.0.1:50132/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"claude-sonnet-5","messages":[{"role":"user","content":"Say ok"}]}' | jq "{model, content: .choices[0].message.content, usage}"
  2.  {
       "model": "claude-sonnet-5",
       "content": "Ok",
       "usage": {
         "completion_tokens": 24,
         "prompt_tokens": 10,
         "total_tokens": 34,
         "completion_tokens_details": {
           "reasoning_tokens": 18,
           "text_tokens": 6
         },
         "prompt_tokens_details": {
           "cached_tokens": 0,
           "text_tokens": 10,
           "cache_write_tokens": 0,
           "cache_creation_tokens": 0,
           "cache_creation_token_details": {
             "ephemeral_5m_input_tokens": 0,
             "ephemeral_1h_input_tokens": 0
           }
         },
         "cache_creation_input_tokens": 0,
         "cache_read_input_tokens": 0,
         "inference_geo": "global",
         "service_tier": "standard"
       }
     }

files: list and upload validation

  1. curl -sS -w "\nHTTP %{http_code}\n" "http://127.0.0.1:50132/v1/files?target_model_names=gpt-5-mini,gpt-5-nano" -H "Authorization: Bearer $LITELLM_MASTER_KEY"
  2.  {"error":{"message":"target_model_names on list files must be a list of one model name. Example: ['gpt-4o']","type":"invalid_request_error","param":null,"code":"400"}}
     HTTP 400
    
  3. curl -sS -w "\nHTTP %{http_code}\n" http://127.0.0.1:50132/v1/files -H "Authorization: Bearer $LITELLM_MASTER_KEY" -F purpose=batch -F target_model_names=gpt-5-mini -F "expires_after[anchor]=created_at" -F file=@batch.jsonl
  4.  {"error":{"message":"{'error': 'Both expires_after[anchor] and expires_after[seconds] must be provided if expires_after is specified'}","type":"invalid_request_error","param":null,"code":"400"}}
     HTTP 400
    

files: deleted file (get, delete, content)

  1. FILE_ID=$(curl -sS http://127.0.0.1:50132/v1/files -H "Authorization: Bearer $LITELLM_MASTER_KEY" -F purpose=batch -F target_model_names=gpt-5-mini -F file=@batch.jsonl | jq -r .id); echo "$FILE_ID"
  2.  bGl0ZWxsbV9wcm94eTphcHBsaWNhdGlvbi9vY3RldC1zdHJlYW07dW5pZmllZF9pZCw5NDUzNWY4Ni00Zjc4LTRlYzQtODM5Zi05Y2FkNzgxMWEyMzQ7dGFyZ2V0X21vZGVsX25hbWVzLGdwdC01LW1pbmk7bGxtX291dHB1dF9maWxlX2lkLGZpbGUtTkw3cGFmQVJxc3JpS0pzSzg2ZlhySDtsbG1fb3V0cHV0X2ZpbGVfbW9kZWxfaWQsZTBhZGMxZDg3NWQwNjIwMTA1MjEzYjAwYWI1OWYxNTQyYzg0OWU4ZTNlMWZjNDQ4MDljYzAzZjMyZmIxYzkyYw
    
  3. curl -sS -w "\nHTTP %{http_code}\n" -X DELETE "http://127.0.0.1:50132/v1/files/$FILE_ID" -H "Authorization: Bearer $LITELLM_MASTER_KEY"
  4.  {"id":"bGl0ZWxsbV9wcm94eTphcHBsaWNhdGlvbi9vY3RldC1zdHJlYW07dW5pZmllZF9pZCw5NDUzNWY4Ni00Zjc4LTRlYzQtODM5Zi05Y2FkNzgxMWEyMzQ7dGFyZ2V0X21vZGVsX25hbWVzLGdwdC01LW1pbmk7bGxtX291dHB1dF9maWxlX2lkLGZpbGUtTkw3cGFmQVJxc3JpS0pzSzg2ZlhySDtsbG1fb3V0cHV0X2ZpbGVfbW9kZWxfaWQsZTBhZGMxZDg3NWQwNjIwMTA1MjEzYjAwYWI1OWYxNTQyYzg0OWU4ZTNlMWZjNDQ4MDljYzAzZjMyZmIxYzkyYw","bytes":157,"object":"file","status":"uploaded","purpose":"batch","filename":"modified_file.jsonl","created_at":1788904970,"expires_at":1791496970,"status_details":null}
     HTTP 200
    
  5. curl -sS -w "\nHTTP %{http_code}\n" "http://127.0.0.1:50132/v1/files/$FILE_ID" -H "Authorization: Bearer $LITELLM_MASTER_KEY"
  6.  {"error":{"message":"File not found: bGl0ZWxsbV9wcm94eTphcHBsaWNhdGlvbi9vY3RldC1zdHJlYW07dW5pZmllZF9pZCw5NDUzNWY4Ni00Zjc4LTRlYzQtODM5Zi05Y2FkNzgxMWEyMzQ7dGFyZ2V0X21vZGVsX25hbWVzLGdwdC01LW1pbmk7bGxtX291dHB1dF9maWxlX2lkLGZpbGUtTkw3cGFmQVJxc3JpS0pzSzg2ZlhySDtsbG1fb3V0cHV0X2ZpbGVfbW9kZWxfaWQsZTBhZGMxZDg3NWQwNjIwMTA1MjEzYjAwYWI1OWYxNTQyYzg0OWU4ZTNlMWZjNDQ4MDljYzAzZjMyZmIxYzkyYw","type":"invalid_request_error","param":null,"code":"404"}}
     HTTP 404
    
  7. curl -sS -w "\nHTTP %{http_code}\n" -X DELETE "http://127.0.0.1:50132/v1/files/$FILE_ID" -H "Authorization: Bearer $LITELLM_MASTER_KEY"
  8.  {"error":{"message":"File not found: bGl0ZWxsbV9wcm94eTphcHBsaWNhdGlvbi9vY3RldC1zdHJlYW07dW5pZmllZF9pZCw5NDUzNWY4Ni00Zjc4LTRlYzQtODM5Zi05Y2FkNzgxMWEyMzQ7dGFyZ2V0X21vZGVsX25hbWVzLGdwdC01LW1pbmk7bGxtX291dHB1dF9maWxlX2lkLGZpbGUtTkw3cGFmQVJxc3JpS0pzSzg2ZlhySDtsbG1fb3V0cHV0X2ZpbGVfbW9kZWxfaWQsZTBhZGMxZDg3NWQwNjIwMTA1MjEzYjAwYWI1OWYxNTQyYzg0OWU4ZTNlMWZjNDQ4MDljYzAzZjMyZmIxYzkyYw","type":"invalid_request_error","param":null,"code":"404"}}
     HTTP 404
    
  9. curl -sS -w "\nHTTP %{http_code}\n" "http://127.0.0.1:50132/v1/files/$FILE_ID/content" -H "Authorization: Bearer $LITELLM_MASTER_KEY"
  10. {"error":{"message":"File not found: bGl0ZWxsbV9wcm94eTphcHBsaWNhdGlvbi9vY3RldC1zdHJlYW07dW5pZmllZF9pZCw5NDUzNWY4Ni00Zjc4LTRlYzQtODM5Zi05Y2FkNzgxMWEyMzQ7dGFyZ2V0X21vZGVsX25hbWVzLGdwdC01LW1pbmk7bGxtX291dHB1dF9maWxlX2lkLGZpbGUtTkw3cGFmQVJxc3JpS0pzSzg2ZlhySDtsbG1fb3V0cHV0X2ZpbGVfbW9kZWxfaWQsZTBhZGMxZDg3NWQwNjIwMTA1MjEzYjAwYWI1OWYxNTQyYzg0OWU4ZTNlMWZjNDQ4MDljYzAzZjMyZmIxYzkyYw","type":"invalid_request_error","param":null,"code":"404"}}
    HTTP 404
    

files: unknown provider id

  1. curl -sS -w "\nHTTP %{http_code}\n" http://127.0.0.1:50132/v1/files/file-doesnotexist -H "Authorization: Bearer $LITELLM_MASTER_KEY"
  2.  {"error":{"message":"Error code: 404 - {'error': {'message': 'No such File object: file-doesnotexist', 'type': 'invalid_request_error', 'param': 'id', 'code': None}, 'detail': {'message': 'No such File object: file-doesnotexist', 'code': None}}","type":"invalid_request_error","param":"id","code":"404"}}
     HTTP 404
    

rerank

  1. curl -sS -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:50132/v1/rerank -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"no-such-rerank-model","query":"hi","documents":["a","b"]}'
  2.  {"error":{"message":"400: {'error': '/rerank: Invalid model name passed in model=no-such-rerank-model. Call `/v1/models` to view available models for your key.'}","type":"invalid_request_error","param":null,"code":"400"}}
     HTTP 400
    

images

  1. curl -sS -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:50132/v1/images/generations -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"no-such-image-model","prompt":"a cat"}'
  2.  {"error":{"message":"400: {'error': '/image/generations: Invalid model name passed in model=no-such-image-model. Call `/v1/models` to view available models for your key.'}","type":"invalid_request_error","param":null,"code":"400"}}
     HTTP 400
    

realtime

  1. curl -sS -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:50132/v1/realtime/client_secrets -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"session":{"type":"realtime","model":"no-such-realtime"}}'
  2.  {"error":{"message":"400: {'error': '/realtime/client_secrets: Invalid model name passed in model=no-such-realtime. Call `/v1/models` to view available models for your key.'}","type":"invalid_request_error","param":null,"code":"400"}}
     HTTP 400
    

anthropic /v1/messages

  1. curl -sS -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:50132/v1/messages -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"gpt-5-mini","max_tokens":16,"messages":"not-a-list"}'
  2.  {"type":"error","error":{"type":"api_error","message":"string indices must be integers, not 'str'"}}
     HTTP 500
    

pass-through

  1. curl -sS -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:50132/unreachable-upstream -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"hi":"there"}'
  2.  {"error":{"message":"Cannot connect to host 127.0.0.1:1 ssl:<ssl.SSLContext object at 0x10f1dc250> [Connect call failed ('127.0.0.1', 1)]","type":"internal_server_error","param":null,"code":"500"}}
     HTTP 500
    

shared handler (credentials)

  1. curl -sS -w "\nHTTP %{http_code}\n" http://127.0.0.1:50132/credentials/by_name/nope -H "Authorization: Bearer $LITELLM_MASTER_KEY"
  2.  {"error":{"message":"Credential not found. Got credential name: nope","type":"internal_server_error","param":null,"code":"404"}}
     HTTP 404
    

The second pass covers the behaviors the first review round changed. The after cursor case is a regression check: list_files re-raises a ProxyException as is, so it answers the same 400 on both legs. The timeout cases show a 408 keeping invalid_request_error on both routes, where the Before leg's images route answered null. The other in-route rejection fix (a ProxyException raised inside a guard-less tail keeping its 4xx status instead of answering 500) has one trigger on these routes, get_file_content reading a file back from a configured storage backend, which needs a managed file row with storage_backend and storage_url; it is pinned by test_get_file_content_keeps_the_status_of_a_rejection_raised_inside_the_route and the rerank route test test_a_rejection_raised_before_routing_keeps_its_own_status, both of which answer 500 without the fix

Before (82e6b84)

rejection raised inside the route (files list, unknown after cursor)

  1. curl -sS -w "\nHTTP %{http_code}\n" "http://127.0.0.1:$PORT/v1/files?after=file-doesnotexist" -H "Authorization: Bearer $LITELLM_MASTER_KEY"
  2.  {"error":{"message":"Invalid 'after' cursor: no file found with id 'file-doesnotexist'.","type":"invalid_request_error","param":"after","code":"400"}}
     HTTP 400
    

timeout (images, request timeout 0.001)

  1. curl -sS -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:$PORT/v1/images/generations -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"gpt-5-mini","prompt":"a cat","timeout":0.001}'
  2.  {"error":{"message":"litellm.Timeout: APITimeoutError - Request timed out. Error_str: Request timed out.. Received Model Group=gpt-5-mini\nAvailable Model Group Fallbacks=None","type":null,"param":null,"code":"408"}}
     HTTP 408
    

timeout (chat completions, request timeout 0.001)

  1. curl -sS -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:$PORT/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"gpt-5-mini","messages":[{"role":"user","content":"Say ok"}],"timeout":0.001}'
  2.  {"error":{"message":"litellm.Timeout: APITimeoutError - Request timed out. Error_str: Request timed out. - timeout value=0.001, time taken=0.0 seconds\n\nDeployment Info: request_timeout: None\ntimeout: None. Received Model Group=gpt-5-mini\nAvailable Model Group Fallbacks=None","type":"invalid_request_error","param":null,"code":"408"}}
     HTTP 408
    

After (b3bcd71)

rejection raised inside the route (files list, unknown after cursor)

  1. curl -sS -w "\nHTTP %{http_code}\n" "http://127.0.0.1:50132/v1/files?after=file-doesnotexist" -H "Authorization: Bearer $LITELLM_MASTER_KEY"
  2.  {"error":{"message":"Invalid 'after' cursor: no file found with id 'file-doesnotexist'.","type":"invalid_request_error","param":"after","code":"400"}}
     HTTP 400
    

timeout (images, request timeout 0.001)

  1. curl -sS -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:50132/v1/images/generations -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"gpt-5-mini","prompt":"a cat","timeout":0.001}'
  2.  {"error":{"message":"litellm.Timeout: APITimeoutError - Request timed out. Error_str: Request timed out.. Received Model Group=gpt-5-mini\nAvailable Model Group Fallbacks=None","type":"invalid_request_error","param":null,"code":"408"}}
     HTTP 408
    

timeout (chat completions, request timeout 0.001)

  1. curl -sS -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:50132/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"gpt-5-mini","messages":[{"role":"user","content":"Say ok"}],"timeout":0.001}'
  2.  {"error":{"message":"litellm.Timeout: APITimeoutError - Request timed out. Error_str: Request timed out. - timeout value=0.001, time taken=0.01 seconds\n\nDeployment Info: request_timeout: None\ntimeout: None. Received Model Group=gpt-5-mini\nAvailable Model Group Fallbacks=None","type":"invalid_request_error","param":null,"code":"408"}}
     HTTP 408
    

The third pass covers the tails the second review round found untested live: a transcription session for a model the gateway does not serve, a WebRTC call whose client secret is real (issued by OpenAI for gpt-realtime-2.1-mini through the gateway) but whose SDP offer OpenAI rejects, and a pass-through route backed by the built-in Anthropic adapter, given a model the gateway does not serve. The token itself is not printed, only its length

Before (82e6b84)

realtime transcription session (unknown model)

  1. curl -sS -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:$PORT/v1/realtime/transcription_sessions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"input_audio_transcription":{"model":"no-such-transcribe"}}'
  2.  {"error":{"message":"{'error': '/realtime/transcription_sessions: Invalid model name passed in model=no-such-transcribe. Call `/v1/models` to view available models for your key.'}","type":"None","param":"None","code":"400"}}
     HTTP 400
    

realtime calls (real client secret, malformed SDP offer)

  1. TOKEN=$(curl -sS -X POST http://127.0.0.1:$PORT/v1/realtime/client_secrets -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"session":{"type":"realtime","model":"gpt-realtime-2.1-mini"}}' | jq -r .value); echo "token length ${#TOKEN}"
  2.  token length 320
    
  3. curl -sS -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:$PORT/v1/realtime/calls -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/sdp" --data-binary not-an-sdp-offer
  4.  {"error":{"message":"{\n    \"error\": {\n        \"message\": \"Failed to parse offer: failed to unmarshal SDP: sdp: syntax error at pos 1: \\\"o\\\"\",\n        \"type\": \"invalid_request_error\",\n        \"code\": \"invalid_offer\",\n        \"param\": \"\"\n    }\n}. Received Model Group=gpt-realtime-2.1-mini\nAvailable Model Group Fallbacks=None","type":"None","param":"None","code":"400"}}
     HTTP 400
    

adapter pass-through (unknown model)

  1. curl -sS -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:$PORT/anthropic-adapter/v1/messages -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"no-such-model","max_tokens":16,"messages":[{"role":"user","content":"hi"}]}'
  2.  {"error":{"message":"400: {'error': 'completion: Invalid model name passed in model=no-such-model'}","type":"None","param":"None","code":"400"}}
     HTTP 400
    

After (b3bcd71)

realtime transcription session (unknown model)

  1. curl -sS -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:50132/v1/realtime/transcription_sessions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"input_audio_transcription":{"model":"no-such-transcribe"}}'
  2.  {"error":{"message":"{'error': '/realtime/transcription_sessions: Invalid model name passed in model=no-such-transcribe. Call `/v1/models` to view available models for your key.'}","type":"invalid_request_error","param":null,"code":"400"}}
     HTTP 400
    

realtime calls (real client secret, malformed SDP offer)

  1. TOKEN=$(curl -sS -X POST http://127.0.0.1:50132/v1/realtime/client_secrets -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"session":{"type":"realtime","model":"gpt-realtime-2.1-mini"}}' | jq -r .value); echo "token length ${#TOKEN}"
  2.  token length 320
    
  3. curl -sS -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:50132/v1/realtime/calls -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/sdp" --data-binary not-an-sdp-offer
  4.  {"error":{"message":"{\n    \"error\": {\n        \"message\": \"Failed to parse offer: failed to unmarshal SDP: sdp: syntax error at pos 1: \\\"o\\\"\",\n        \"type\": \"invalid_request_error\",\n        \"code\": \"invalid_offer\",\n        \"param\": \"\"\n    }\n}. Received Model Group=gpt-realtime-2.1-mini\nAvailable Model Group Fallbacks=None","type":"invalid_request_error","param":null,"code":"400"}}
     HTTP 400
    

adapter pass-through (unknown model)

  1. curl -sS -w "\nHTTP %{http_code}\n" -X POST http://127.0.0.1:50132/anthropic-adapter/v1/messages -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"no-such-model","max_tokens":16,"messages":[{"role":"user","content":"hi"}]}'
  2.  {"error":{"message":"400: {'error': 'completion: Invalid model name passed in model=no-such-model'}","type":"invalid_request_error","param":null,"code":"400"}}
     HTTP 400
    

Observations the run turned up, none caused or worsened by this PR:

Type

🐛 Bug Fix

Caveats (if any)

Medium

Low

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR
  • edde951 passes /live-pr-risk
  • 8b89c90 passes /live-pr-risk
  • b3bcd71 passes /live-pr-risk

… param

The proxy's exception tails defaulted `type` and `param` to the four-character
string "None", which is neither a known OpenAI error type nor the JSON null the
nullable `param` field is typed as, so a client's error handler matched nothing
and fell into its generic branch.

Lifts the helpers PR #39521 added for the unified LLM endpoints into
litellm/proxy/common_utils/openai_error_payload.py and calls them from the file,
rerank, image, realtime, anthropic, and pass-through route families, plus the
shared handle_exception_on_proxy handler that the management, batches,
fine-tuning, credential, SCIM, guardrail, and customer routes funnel through.

The remaining families (proxy_server, auth, health, spend tracking, and
management endpoints) follow in separate PRs so each slice stays QA'able on a
live proxy.
@greptile-apps

greptile-apps Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR centralizes OpenAI-compatible proxy error shaping and applies it across files, rerank, images, realtime, Anthropic, pass-through, streaming, and shared exception handlers.

  • Replaces literal "None" values with status-derived error types and nullable parameters.
  • Preserves status codes carried by HTTPException and ProxyException.
  • Adds regression coverage for affected endpoint tails, SSE payloads, and shared helpers.
  • No changes were made since the previous review.

Confidence Score: 5/5

The PR appears safe to merge; no actionable new issue or outstanding previous finding remains.

All previous threads were resolved or correctly withdrawn, the current head is unchanged since the previous review, and the full diff introduces no confirmed rule violation or merge-blocking behavior.

Important Files Changed

Filename Overview
litellm/proxy/common_utils/openai_error_payload.py Introduces shared helpers for preserving exception status codes and producing OpenAI-compatible error type and parameter fields.
litellm/proxy/common_request_processing.py Reuses the shared error helpers in HTTP exception conversion, SSE errors, and request-processing failure paths.
litellm/proxy/openai_files_endpoints/files_endpoints.py Normalizes error payloads and preserves carried statuses across all files endpoint exception tails.
litellm/proxy/realtime_endpoints/endpoints.py Applies consistent error typing and nullable parameters to realtime client-secret, WebRTC, and transcription failures.
litellm/proxy/pass_through_endpoints/pass_through_endpoints.py Normalizes pass-through error payloads while retaining exception status codes and custom headers.
tests/test_litellm/proxy/common_utils/test_openai_error_payload.py Covers status-to-type mapping, nullable parameters, carried fields, and stringified ProxyException status codes.
tests/test_litellm/proxy/openai_files_endpoint/test_files_endpoint.py Adds route-level regressions for validation failures, missing managed files, and in-route status preservation.

Reviews (7): Last reviewed commit: "test(proxy): type the realtime WebRTC fi..." | Re-trigger Greptile

Comment thread tests/test_litellm/proxy/common_utils/test_openai_error_payload.py
@codecov

codecov Bot commented Sep 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@mateo-berri

Copy link
Copy Markdown
Contributor Author

@greptileai

Base automatically changed from litellm_fix_guardrail_error_stringified_none to litellm_internal_staging September 3, 2026 21:36
…itellm_openai_error_payload_non_llm_routes

# Conflicts:
#	litellm/proxy/anthropic_endpoints/endpoints.py
#	litellm/proxy/image_endpoints/endpoints.py
@mateo-berri
mateo-berri requested a review from a team September 8, 2026 18:11
@codspeed

codspeed Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing litellm_openai_error_payload_non_llm_routes (b3bcd71) with litellm_internal_staging (35451ec)

Open in CodSpeed

@mateo-berri mateo-berri added run-ci and removed run-ci labels Sep 8, 2026
@mateo-berri

Copy link
Copy Markdown
Contributor Author

@greptileai

…AI error payload

error_status_code only read status_code, so a ProxyException raised
before routing (which stores its status as the string code) answered
500 with its 4xx type through the rerank, images, realtime, files, and
pass-through tails. It now falls back to a decimal code. A 408 maps to
timeout_error instead of invalid_request_error.

Tail regressions for rerank, images, realtime calls, and the chat
pass-through fail at the merge base with ('None', 'None'); the new
files-test helpers are fully typed.
@mateo-berri

Copy link
Copy Markdown
Contributor Author

@greptileai

@mateo-berri mateo-berri added run-ci and removed run-ci labels Sep 8, 2026
@mateo-berri

Copy link
Copy Markdown
Contributor Author

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

@mateo-berri

Copy link
Copy Markdown
Contributor Author

@greptileai

Comment thread litellm/proxy/common_utils/openai_error_payload.py
Comment thread tests/test_litellm/proxy/realtime_endpoints/test_realtime_webrtc_endpoints.py Outdated
@mateo-berri mateo-berri added run-ci and removed run-ci labels Sep 8, 2026

@ParBproject ParBproject left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There’s one edge case in the shared helper that seems to leave the original bug reachable: openai_error_type() accepts any str as authoritative, including the literal string "None". That means any existing ProxyException/provider exception that already carries type="None" will still serialize "type": "None", even though this module’s stated contract is to eliminate that value. The new tests only cover an exception with no type attribute / None (non-string), so they won’t catch a legacy string sentinel. I’d suggest treating at least "None" (and probably empty strings) as missing and falling back to the status mapping, with a regression test such as a carrier whose type = "None". The same consideration may apply to openai_error_param() if legacy exceptions carry param="None".

@mateo-berri mateo-berri added run-ci and removed run-ci labels Sep 8, 2026
@mateo-berri

Copy link
Copy Markdown
Contributor Author

@greptileai

Comment thread tests/test_litellm/proxy/image_endpoints/test_endpoints.py
@mateo-berri

Copy link
Copy Markdown
Contributor Author

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit b3bcd71. Configure here.

@mateo-berri

Copy link
Copy Markdown
Contributor Author

@greptileai

@mateo-berri
mateo-berri merged commit 2b9a69d into litellm_internal_staging Sep 8, 2026
133 of 135 checks passed
@mateo-berri
mateo-berri deleted the litellm_openai_error_payload_non_llm_routes branch September 8, 2026 23:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: All five /v1/files routes ship type and param as the literal string "None", so every error body is unclassifiable

3 participants