fix(proxy): stop answering with the literal string "None" on the proxy-server, auth, and health routes

[mateo-berri]

TLDR

Problem this solves:

• proxy_server.py answers "type": "None" on 63 error sites
• Auth rejections and health checks ship "param": "None" too
• Those are four-character strings, not OpenAI error types
• param is typed nullable, so a string is plain wrong there

How it solves it:

• Calls the shared helpers fix(proxy): stop shipping the literal string "None" as error type and param #39536 added, no second convention
• type falls back to what the HTTP status stands for
• A missing param serializes as JSON null
• Covers proxy_server routes, auth rejections, health services
• Still outstanding: spend tracking and management endpoints

User Flow

Before: a developer whose app talks to the gateway gets errors no OpenAI SDK can classify, so every failure on these routes lands in their catch-all branch

1. They send POST https://litellm-domain/v1/chat/completions with no Authorization header
2. HTTP 401 comes back as {"error":{"message":"Authentication Error, No api key passed in.","type":"auth_error","param":"None","code":"401"}}, so their handler reads param as the four-character string None where it expected a JSON null and renders "problem with field None"
3. They send POST https://litellm-domain/v1/audio/transcriptions with model=no-such-audio-model
4. HTTP 400 comes back with "type":"None", so their branch on invalid_request_error never fires and the request is reported as an unknown error with no guidance
5. They send GET https://litellm-domain/v1/assistants?custom_llm_provider=openai at a gateway with no assistants provider configured
6. HTTP 500 comes back with "type":"None" and "param":"None", so their retry logic cannot tell a server fault from a bad request
7. They check GET https://litellm-domain/health/services?service=bogus_service and get HTTP 400 with a real "type":"auth_error" but "param":"None" again

After: the same requests come back with a real OpenAI error type and a JSON null param, so the handler they already wrote classifies them

1. They send POST https://litellm-domain/v1/chat/completions with no Authorization header
2. HTTP 401 comes back as {"error":{"message":"Authentication Error, No api key passed in.","type":"auth_error","param":null,"code":"401"}}, so their handler reads param as null and correctly reports that no single field was named
3. They send POST https://litellm-domain/v1/audio/transcriptions with model=no-such-audio-model
4. HTTP 400 comes back with "type":"invalid_request_error", so their branch fires and the caller sees a "fix your request" message
5. They send GET https://litellm-domain/v1/assistants?custom_llm_provider=openai at a gateway with no assistants provider configured
6. HTTP 500 comes back with "type":"internal_server_error" and "param":null, so their retry logic can tell a server fault from a bad request
7. GET https://litellm-domain/health/services?service=bogus_service still returns HTTP 400 with "type":"auth_error", now with "param":null

Relevant issues

Linear ticket

Part of LIT-6829

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Both legs run the same proxy, two uvicorn workers each, against the same Postgres, and differ only in the commit they were booted from. bede8b5ea4 is this PR's merge base, the commit it branches from on #39536; 32074cf4de is this PR's tip.

Config used on both legs:

model_list:
  - model_name: gpt-4o-mini
    litellm_params:
      model: openai/gpt-4o-mini
      api_key: os.environ/OPENAI_API_KEY

general_settings:
  master_key: sk-1234
  enable_jwt_auth: true
  pass_through_endpoints:
    - path: "/unreachable-upstream"
      target: "http://127.0.0.1:1/nope"
      headers:
        content-type: application/json

litellm_settings:
  drop_params: true

Boot on each leg (ports differ, <port> is 20091 before and 46742 after):

./.venv/bin/python litellm/proxy/proxy_cli.py --config lit6829_config.yaml --port <port> --num_workers 2

Before (bede8b5)

Assistants with no assistants provider configured

1. Run the request

curl -sS -X GET "http://127.0.0.1:20091/v1/assistants?custom_llm_provider=openai" -H 'Authorization: Bearer sk-1234'

2. Observe type and param as the string None

{"error":{"message":"'custom_llm_provider' must be set. Either via:\n `Router(assistants_config={'custom_llm_provider': ..})` \nor\n `router.arun_thread(custom_llm_provider=..)`","type":"None","param":"None","code":"500"}}

Audio transcription with an unknown model

1. Run the request

curl -sS -X POST "http://127.0.0.1:20091/v1/audio/transcriptions" -H 'Authorization: Bearer sk-1234' -F 'model=no-such-audio-model' -F 'file=@/etc/hosts'

2. Observe type and param as the string None

{"error":{"message":"{'error': '/audio/transcriptions: Invalid model name passed in model=no-such-audio-model. Call `/v1/models` to view available models for your key.'}","type":"None","param":"None","code":"400"}}

Health services with an unknown service

1. Run the request

curl -sS -X GET "http://127.0.0.1:20091/health/services?service=bogus_service" -H 'Authorization: Bearer sk-1234'

2. Observe a real type but param as the string None

{"error":{"message":"{'error': \"Service must be in list. Service=bogus_service not in typing.Union[typing.Literal['slack_budget_alerts', 'langfuse', 'langfuse_otel', 'slack', 'ms_teams', 'openmeter', 'webhook', 'email', 'braintrust', 'datadog', 'datadog_llm_observability', 'generic_api', 'arize', 'galileo', 'newrelic', 'sqs'], str]\"}","type":"auth_error","param":"None","code":"400"}}

Chat completion with no API key

1. Run the request

curl -sS -X POST "http://127.0.0.1:20091/v1/chat/completions" -H 'Content-Type: application/json' -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"hi"}]}'

2. Observe param as the string None

{"error":{"message":"Authentication Error, No api key passed in.","type":"auth_error","param":"None","code":"401"}}

Control: a real chat completion still works

1. Run the request

curl -sS -X POST "http://127.0.0.1:20091/v1/chat/completions" -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Say OK"}],"max_tokens":5}'

2. Observe a real completion from OpenAI

{"id":"chatcmpl-EJyeQ5uib5jz9Y61fpGAHqduCPeVU","created":1788430082,"model":"gpt-4o-mini","object":"chat.completion","choices":[{"finish_reason":"stop","index":0,"message":{"content":"OK!","role":"assistant"}}],"usage":{"completion_tokens":2,"prompt_tokens":9,"total_tokens":11}}

After (32074cf)

Assistants with no assistants provider configured

1. Run the request

curl -sS -X GET "http://127.0.0.1:46742/v1/assistants?custom_llm_provider=openai" -H 'Authorization: Bearer sk-1234'

2. Observe a real type and a JSON null param

{"error":{"message":"'custom_llm_provider' must be set. Either via:\n `Router(assistants_config={'custom_llm_provider': ..})` \nor\n `router.arun_thread(custom_llm_provider=..)`","type":"internal_server_error","param":null,"code":"500"}}

Audio transcription with an unknown model

1. Run the request

curl -sS -X POST "http://127.0.0.1:46742/v1/audio/transcriptions" -H 'Authorization: Bearer sk-1234' -F 'model=no-such-audio-model' -F 'file=@/etc/hosts'

2. Observe a real type and a JSON null param

{"error":{"message":"{'error': '/audio/transcriptions: Invalid model name passed in model=no-such-audio-model. Call `/v1/models` to view available models for your key.'}","type":"invalid_request_error","param":null,"code":"400"}}

Health services with an unknown service

1. Run the request

curl -sS -X GET "http://127.0.0.1:46742/health/services?service=bogus_service" -H 'Authorization: Bearer sk-1234'

2. Observe the same type with a JSON null param

{"error":{"message":"{'error': \"Service must be in list. Service=bogus_service not in typing.Union[typing.Literal['slack_budget_alerts', 'langfuse', 'langfuse_otel', 'slack', 'ms_teams', 'openmeter', 'webhook', 'email', 'braintrust', 'datadog', 'datadog_llm_observability', 'generic_api', 'arize', 'galileo', 'newrelic', 'sqs'], str]\"}","type":"auth_error","param":null,"code":"400"}}

Chat completion with no API key

1. Run the request

curl -sS -X POST "http://127.0.0.1:46742/v1/chat/completions" -H 'Content-Type: application/json' -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"hi"}]}'

2. Observe a JSON null param

{"error":{"message":"Authentication Error, No api key passed in.","type":"auth_error","param":null,"code":"401"}}

Control: a real chat completion still works

1. Run the request

curl -sS -X POST "http://127.0.0.1:46742/v1/chat/completions" -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Say OK"}],"max_tokens":5}'

2. Observe a real completion from OpenAI

{"id":"chatcmpl-EJyhAkxsDATmI2HrPfglA7mjix89e","created":1788430252,"model":"gpt-4o-mini","object":"chat.completion","choices":[{"finish_reason":"stop","index":0,"message":{"content":"OK!","role":"assistant"}}],"usage":{"completion_tokens":2,"prompt_tokens":9,"total_tokens":11}}

Observations from the run:

• param was wrong on more routes than type was
• /v1/assistants answers 500 for a caller's config mistake
• POST /v1/audio/speech answers {"detail": ...}, not an error object
• Both untouched here, neither caused nor worsened by this PR

Type

🐛 Bug Fix

Caveats (if any)

Low

• Stacked on fix(proxy): stop shipping the literal string "None" as error type and param #39536, so it merges after that one
• The unreachable-JWKS 503 was not proven on a live proxy
  • It only fires behind JWT auth, which needs an enterprise license
  • The license in the local env fails is_premium(), so no run reaches it
  • Reverting that line fails test_jwks_unavailable_exception_is_openai_shaped
  • The same one-literal change is proven live on four routes here
• Spend and management routes are fix(proxy): stop answering with the literal string "None" on the spend and management routes #39542, stacked on this one
• Seven nested code=getattr(e, "code", ...) reads left as they were

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[greptile-apps]

Greptile Summary

This PR normalizes proxy, authentication, streaming, assistants, audio, and health-route error payloads so missing parameters serialize as JSON null and missing error types derive from HTTP status.

• Reuses shared OpenAI error-payload helpers across proxy exception conversion paths.
• Adds regression coverage for authentication, unavailable JWKS, health-service rejection, and assistants errors.
• Preserves existing status codes while replacing literal "None" values.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
litellm/proxy/auth/auth_exception_handler.py	Authentication failures now serialize absent error parameters as null while preserving existing classifications and status codes.
litellm/proxy/auth/handle_jwt.py	Unavailable-JWKS errors now expose a nullable parameter without changing authentication behavior.
litellm/proxy/health_endpoints/_health_endpoints.py	Health-service exception conversion now uses the shared nullable-parameter normalization helper.
litellm/proxy/proxy_server.py	Proxy endpoint and streaming exception paths now derive stable OpenAI error types and nullable parameters through shared helpers.
tests/test_litellm/proxy/auth/test_auth_exception_handler.py	Adds serialization checks for HTTP, database-unavailable, and generic authentication failures.
tests/test_litellm/proxy/auth/test_handle_jwt.py	Adds coverage for the OpenAI-compatible unavailable-JWKS error payload.
tests/test_litellm/proxy/health_endpoints/test_health_endpoints.py	The unknown-service test reaches the modified HTTPException conversion branch and verifies a null parameter.
tests/test_litellm/proxy/test_proxy_server.py	Adds route-level coverage for assistants error type and parameter serialization.

_{Reviews (2): Last reviewed commit: "fix(proxy): stop answering with the lite..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[mateo-berri]

@greptileai