fix(openai): mint workload identity tokens for PrivateLink and regional api.openai.com hosts

[mateo-berri]

TLDR

Problem this solves:

• OpenAI workload identity only fires when the host is exactly api.openai.com
• PrivateLink (<region>.privatelink.api.openai.com) and regional (eu.api.openai.com) hosts get no token
• /v1/chat/completions then fails locally with "The api_key client option must be set"
• /v1/responses and /v1/messages go out with no bearer and get 401 back

How it solves it:

• The WIF host check now uses is_openai_backed_api_base from fix(openai): default stream usage on PrivateLink and regional api.openai.com hosts #39614
• api.openai.com and any *.api.openai.com host qualify; plaintext http:// still does not
• Lookalikes such as api.openai.com.evil.example still get no token
• Regression tests cover PrivateLink, regional, lookalike, and plaintext hosts on both auth paths

User Flow

Before: a proxy admin who points a keyless openai/ deployment at their PrivateLink endpoint gets an auth failure on every call, while the same setup against api.openai.com works

1. The admin sets OPENAI_IDENTITY_PROVIDER_ID, OPENAI_SERVICE_ACCOUNT_ID, and OPENAI_IDENTITY_TOKEN_FILE, adds an openai/gpt-5.6 deployment with api_base: https://southcentralus.privatelink.api.openai.com/v1 and no api_key, and starts the proxy
2. They send POST https://litellm-domain/v1/chat/completions with {"model": "gpt-5.6-privatelink", "messages": [{"role": "user", "content": "Reply with exactly the word: pong"}]}
3. They get HTTP 500 with The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable, and nothing ever calls https://auth.openai.com/oauth/token
4. They send POST https://litellm-domain/v1/responses with {"model": "gpt-5.6-privatelink", "input": "Reply with exactly the word: pong"} and get HTTP 401 Incorrect API key provided: None, because the request reached their endpoint with no bearer token
5. They send POST https://litellm-domain/v1/messages with {"model": "gpt-5.6-privatelink", "max_tokens": 20, "messages": [...]} and get the same HTTP 401 Incorrect API key provided: None
6. They send the same three requests to a deployment with no api_base and get HTTP 200 pong from every one, so the credentials are fine and only the hostname differs

After: the same PrivateLink deployment exchanges the identity token and answers on all three endpoints

1. The admin sets OPENAI_IDENTITY_PROVIDER_ID, OPENAI_SERVICE_ACCOUNT_ID, and OPENAI_IDENTITY_TOKEN_FILE, adds an openai/gpt-5.6 deployment with api_base: https://southcentralus.privatelink.api.openai.com/v1 and no api_key, and starts the proxy
2. They send POST https://litellm-domain/v1/chat/completions with {"model": "gpt-5.6-privatelink", "messages": [{"role": "user", "content": "Reply with exactly the word: pong"}]}
3. They get HTTP 200 with "content": "pong"; the proxy exchanged the identity token at https://auth.openai.com/oauth/token first and sent the request to the PrivateLink endpoint with that bearer
4. They send POST https://litellm-domain/v1/responses with the same model and get HTTP 200 with "status": "completed" and output text pong
5. They send POST https://litellm-domain/v1/messages with the same model and get HTTP 200 with "stop_reason": "end_turn" and text pong
6. They send the same three requests to the deployment with no api_base and still get HTTP 200 pong from every one

Relevant issues

Linear ticket

Resolves LIT-6902

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Shared setup for both legs. The proxy boots from the named commit with python litellm/proxy/proxy_cli.py --config config.yaml --port 48007 --num_workers 2 --detailed_debug. Its environment carries a real OpenAI identity provider and service account (OPENAI_IDENTITY_PROVIDER_ID, OPENAI_SERVICE_ACCOUNT_ID, OPENAI_IDENTITY_TOKEN_FILE pointing at a freshly minted subject token) and no OPENAI_API_KEY or OPENAI_BASE_URL anywhere, so every 200 below is a real gpt-5.6 call paid for through the exchanged token

model_list:
  - model_name: gpt-5.6-privatelink
    litellm_params:
      model: openai/gpt-5.6
      api_base: https://southcentralus.privatelink.api.openai.com/v1
  - model_name: gpt-5.6-eu
    litellm_params:
      model: openai/gpt-5.6
      api_base: https://eu.api.openai.com/v1
  - model_name: gpt-5.6-default
    litellm_params:
      model: openai/gpt-5.6
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
router_settings:
  disable_cooldowns: true

disable_cooldowns keeps the eu deployment's expected 4xx on one endpoint from cooling it down before the next endpoint's call. PrivateLink hostnames only resolve inside the customer's VPC, so the proxy runs with HTTPS_PROXY pointed at a local CONNECT forwarder that terminates TLS for southcentralus.privatelink.api.openai.com with a locally trusted CA (SSL_CERT_FILE) and relays every byte to api.openai.com, tunneling every other host (auth.openai.com, eu.api.openai.com) untouched. Its only edit is the Host header, because Cloudflare in front of api.openai.com answers 403 to a host it does not serve; the deployment keeps the genuine PrivateLink URL, which is all the auth decision reads. eu.api.openai.com resolves publicly and is reached directly; the test org has no EU data-residency project, so the After answer there is OpenAI's "only accessible by projects with geography restrictions enabled", an authenticated rejection, where Before got "Incorrect API key provided: None" for a request that carried no bearer at all. The forwarder log lines quoted under each case show which hosts were actually contacted

Every step below runs the same three commands, one per deployment, with MODEL swapped in:

curl -s -w '\nHTTP %{http_code}\n' http://127.0.0.1:48007/v1/chat/completions \
  -H 'Authorization: Bearer sk-lit6902' -H 'Content-Type: application/json' \
  -d '{"model": "MODEL", "messages": [{"role": "user", "content": "Reply with exactly the word: pong"}], "max_tokens": 20}'

curl -s -w '\nHTTP %{http_code}\n' http://127.0.0.1:48007/v1/responses \
  -H 'Authorization: Bearer sk-lit6902' -H 'Content-Type: application/json' \
  -d '{"model": "MODEL", "input": "Reply with exactly the word: pong", "max_output_tokens": 20}'

curl -s -w '\nHTTP %{http_code}\n' http://127.0.0.1:48007/v1/messages \
  -H 'Authorization: Bearer sk-lit6902' -H 'Content-Type: application/json' \
  -d '{"model": "MODEL", "max_tokens": 20, "messages": [{"role": "user", "content": "Reply with exactly the word: pong"}]}'

Before (8699998)

POST /v1/chat/completions

1. MODEL=gpt-5.6-privatelink

   {"error":{"message":"litellm.AuthenticationError: AuthenticationError: OpenAIException - The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable. Received Model Group=gpt-5.6-privatelink\nAvailable Model Group Fallbacks=None","type":null,"param":null,"code":"500"}}
   HTTP 500
   

   The forwarder log shows no CONNECT auth.openai.com:443 and no CONNECT southcentralus.privatelink.api.openai.com:443 for this call: the request never left the proxy

2. MODEL=gpt-5.6-eu

   {"error":{"message":"litellm.AuthenticationError: AuthenticationError: OpenAIException - The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable. Received Model Group=gpt-5.6-eu\nAvailable Model Group Fallbacks=None","type":null,"param":null,"code":"500"}}
   HTTP 500
   

3. MODEL=gpt-5.6-default

   {"id":"chatcmpl-EK9VzT7khNbdnXK4Upewxw0tZ4dGA","created":1788471843,"model":"gpt-5.6-default","object":"chat.completion","choices":[{"finish_reason":"stop","index":0,"message":{"content":"pong","role":"assistant",…}}],"usage":{"completion_tokens":4,"prompt_tokens":13,"total_tokens":17,…}}
   HTTP 200
   

   Forwarder log: 14:44:02 CONNECT auth.openai.com:443 then 14:44:02 CONNECT api.openai.com:443

POST /v1/responses

1. MODEL=gpt-5.6-privatelink

   {"error":{"message":"litellm.AuthenticationError: AuthenticationError: OpenAIException - {\n  \"error\": {\n    \"message\": \"Incorrect API key provided: None. You can find your API key at https://platform.openai.com/account/api-keys.\",\n    \"type\": \"invalid_request_error\",\n    \"param\": null,\n    \"code\": \"invalid_api_key\"\n  }\n}. Received Model Group=gpt-5.6-privatelink\nAvailable Model Group Fallbacks=None","type":null,"param":null,"code":"401"}}
   HTTP 401
   

   Forwarder log: 14:44:00 CONNECT southcentralus.privatelink.api.openai.com:443 -> MITM, relaying bytes to api.openai.com:443 with no CONNECT auth.openai.com:443 before it: the request went out with no bearer

2. MODEL=gpt-5.6-eu

   {"error":{"message":"litellm.AuthenticationError: AuthenticationError: OpenAIException - {\n  \"error\": {\n    \"message\": \"Incorrect API key provided: None. You can find your API key at https://platform.openai.com/account/api-keys.\",\n    \"type\": \"invalid_request_error\",\n    \"param\": null,\n    \"code\": \"invalid_api_key\"\n  }\n}. Received Model Group=gpt-5.6-eu\nAvailable Model Group Fallbacks=None","type":null,"param":null,"code":"401"}}
   HTTP 401
   

   Forwarder log: 14:44:01 CONNECT eu.api.openai.com:443 -> raw tunnel to the real host, again with no token exchange before it

3. MODEL=gpt-5.6-default

   {"id":"resp_v7hUUau0ztqcjrZCxmF3FoW6lRFj9qHLDPt4Pw_K…","model":"gpt-5.6-default","object":"response","output":[{"content":[{"text":"pong","type":"output_text",…}],"role":"assistant","status":"completed","type":"message",…}],"status":"completed","usage":{"input_tokens":13,"output_tokens":5,"total_tokens":18,…},…}
   HTTP 200
   

POST /v1/messages

1. MODEL=gpt-5.6-privatelink

   {"error":{"message":"litellm.AuthenticationError: AuthenticationError: OpenAIException - {\n  \"error\": {\n    \"message\": \"Incorrect API key provided: None. You can find your API key at https://platform.openai.com/account/api-keys.\",\n    \"type\": \"invalid_request_error\",\n    \"param\": null,\n    \"code\": \"invalid_api_key\"\n  }\n}. Received Model Group=gpt-5.6-privatelink\nAvailable Model Group Fallbacks=None","type":null,"param":null,"code":"401"}}
   HTTP 401
   

2. MODEL=gpt-5.6-eu

   {"error":{"message":"litellm.AuthenticationError: AuthenticationError: OpenAIException - {\n  \"error\": {\n    \"message\": \"Incorrect API key provided: None. You can find your API key at https://platform.openai.com/account/api-keys.\",\n    \"type\": \"invalid_request_error\",\n    \"param\": null,\n    \"code\": \"invalid_api_key\"\n  }\n}. Received Model Group=gpt-5.6-eu\nAvailable Model Group Fallbacks=None","type":null,"param":null,"code":"401"}}
   HTTP 401
   

3. MODEL=gpt-5.6-default

   {"id":"resp_bGl0ZWxsbTpjdXN…","type":"message","role":"assistant","model":"gpt-5.6-default","content":[{"type":"text","text":"pong"}],"stop_reason":"end_turn","usage":{"input_tokens":13,"output_tokens":5,…}}
   HTTP 200
   

After (19da217)

POST /v1/chat/completions

1. MODEL=gpt-5.6-privatelink

   {"id":"chatcmpl-EK9WSv9dymMFty2F3tprA6N0ZKx1G","created":1788471872,"model":"gpt-5.6-privatelink","object":"chat.completion","choices":[{"finish_reason":"stop","index":0,"message":{"content":"pong","role":"assistant",…}}],"usage":{"completion_tokens":4,"prompt_tokens":13,"total_tokens":17,…}}
   HTTP 200
   

   Forwarder log: 14:44:31 CONNECT auth.openai.com:443 (token exchange) then 14:44:31 CONNECT southcentralus.privatelink.api.openai.com:443 -> MITM, relaying bytes to api.openai.com:443 and rewrote Host header to api.openai.com on POST /v1/chat/completions

2. MODEL=gpt-5.6-eu

   {"error":{"message":"litellm.BadRequestError: OpenAIException - This endpoint is only accessible by projects with geography restrictions enabled.. Received Model Group=gpt-5.6-eu\nAvailable Model Group Fallbacks=None","type":"invalid_request_error","param":null,"code":"403"}}
   HTTP 403
   

   Forwarder log: 14:44:38 CONNECT auth.openai.com:443 then 14:44:38 CONNECT eu.api.openai.com:443. OpenAI now evaluates the authenticated project's residency instead of rejecting a missing key

3. MODEL=gpt-5.6-default

   {"id":"chatcmpl-EK9WiXdQJbAmfd6TwUC1QqwiVtHvm","created":1788471888,"model":"gpt-5.6-default","object":"chat.completion","choices":[{"finish_reason":"stop","index":0,"message":{"content":"pong","role":"assistant",…}}],"usage":{"completion_tokens":4,"prompt_tokens":13,"total_tokens":17,…}}
   HTTP 200
   

   Forwarder log: 14:44:47 CONNECT auth.openai.com:443 then 14:44:48 CONNECT api.openai.com:443

POST /v1/responses

1. MODEL=gpt-5.6-privatelink

   {"id":"resp_yqMVSRhiqhTZ4Qo…","model":"gpt-5.6-privatelink","object":"response","output":[{"content":[{"text":"pong","type":"output_text",…}],"role":"assistant","status":"completed","type":"message",…}],"status":"completed","usage":{"input_tokens":13,"output_tokens":5,"total_tokens":18,…},…}
   HTTP 200
   

   Forwarder log: 14:44:33 CONNECT auth.openai.com:443 then rewrote Host header to api.openai.com on POST /v1/responses

2. MODEL=gpt-5.6-eu

   {"error":{"message":"litellm.AuthenticationError: AuthenticationError: OpenAIException - {\n  \"error\": {\n    \"message\": \"This endpoint is only accessible by projects with geography restrictions enabled.\",\n    \"type\": null,\n    \"code\": null,\n    \"param\": null\n  },\n  \"status\": 401\n}. Received Model Group=gpt-5.6-eu\nAvailable Model Group Fallbacks=None","type":null,"param":null,"code":"401"}}
   HTTP 401
   

3. MODEL=gpt-5.6-default

   {"id":"resp_N7HsbRl01F3llaDqPHWtJasqWsTuzFpP5hXjYbiV…","model":"gpt-5.6-default","object":"response","output":[{"content":[{"text":"pong","type":"output_text",…}],"role":"assistant","status":"completed","type":"message",…}],"status":"completed","usage":{"input_tokens":13,"output_tokens":5,"total_tokens":18,…},…}
   HTTP 200
   

POST /v1/messages

1. MODEL=gpt-5.6-privatelink

   {"id":"resp_bGl0ZWxsbTpjdXN…","type":"message","role":"assistant","model":"gpt-5.6-privatelink","content":[{"type":"text","text":"pong"}],"stop_reason":"end_turn","usage":{"input_tokens":13,"output_tokens":5,…}}
   HTTP 200
   

   Forwarder log: 14:44:35 CONNECT auth.openai.com:443 then 14:44:36 CONNECT southcentralus.privatelink.api.openai.com:443 -> MITM and rewrote Host header to api.openai.com on POST /v1/responses (the Messages route rides the Responses API upstream)

2. MODEL=gpt-5.6-eu

   {"error":{"message":"litellm.AuthenticationError: AuthenticationError: OpenAIException - {\n  \"error\": {\n    \"message\": \"This endpoint is only accessible by projects with geography restrictions enabled.\",\n    \"type\": null,\n    \"code\": null,\n    \"param\": null\n  },\n  \"status\": 401\n}. Received Model Group=gpt-5.6-eu\nAvailable Model Group Fallbacks=None","type":null,"param":null,"code":"401"}}
   HTTP 401
   

3. MODEL=gpt-5.6-default

   {"id":"resp_bGl0ZWxsbTpjdXN…","type":"message","role":"assistant","model":"gpt-5.6-default","content":[{"type":"text","text":"pong"}],"stop_reason":"end_turn","usage":{"input_tokens":13,"output_tokens":5,…}}
   HTTP 200
   

Observations from the run, none caused by this PR:

• Chat completions reports the missing-key failure as HTTP 500, unchanged here
• eu.api.openai.com answers varied between runs (pong once, geography error later), OpenAI-side

Type

🐛 Bug Fix

Caveats (if any)

Low

• PrivateLink proof uses a local TLS forwarder, not a real VPC endpoint
• The eu leg proves the bearer is sent, not an EU completion
• Chat completions still maps a missing key to HTTP 500, pre-existing
• PR feat(proxy): configure OpenAI workload identity federation from the Add Model and LLM Credentials forms #39613 (merged into the litellm_internal_copy_38013 stack, not on staging yet) carries an equivalent copy of this host check, so that stack will conflict on _targets_openai_api and should keep this version
• ci/circleci: litellm_router_unit_testing fails test_no_linear_scans_in_router on staging too (methods added by feat(router): limit heuristic_v2 auto-routers to one without the auto_router license feature #39468); this PR does not touch router.py
• proxy-extras / Run tests fails four TestMigrateDeployAttemptAccounting cases on staging's last Unit Tests runs too; this PR does not touch proxy-extras
• ci/circleci: e2e_ui_testing fails searchUsers.spec.ts on every branch that includes fix(ui): let the Internal Users search box match user_id as well as email #39604, whose placeholder rename the spec never followed (LIT-6919); the last staging pipeline that passed it predates that merge, and this PR touches no UI. None of the three is a required check

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

• 19da217 passes /live-pr-risk

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_fix_openai_wif_openai_backed_hosts (19da217) with litellm_internal_staging (ab44e8d)}

[greptile-apps]

Greptile Summary

This PR expands OpenAI workload-identity authentication from the default API hostname to HTTPS regional and PrivateLink subdomains while preserving rejection of plaintext and lookalike hosts.

• Reuses the shared OpenAI-backed API-base predicate in workload identity resolution.
• Adds mocked regression coverage for regional and PrivateLink configuration, client construction, and Responses authorization headers.
• Preserves static API-key precedence and existing behavior for the default OpenAI endpoint.

Confidence Score: 5/5

The PR appears safe to merge with the intended workload-identity expansion constrained to HTTPS OpenAI API hostnames.

The shared predicate preserves all previously accepted cases, rejects plaintext and suffix-lookalike domains, and the affected chat and Responses authentication paths receive focused mocked regression coverage.

Important Files Changed

Filename	Overview
litellm/llms/openai/workload_identity.py	Broadens the workload-identity host gate to OpenAI-controlled regional and PrivateLink subdomains while retaining HTTPS and static-key safeguards.
tests/test_litellm/llms/openai/test_openai_workload_identity.py	Adds mocked positive and negative regression cases across configuration resolution, SDK client construction, and Responses bearer generation.

_{Reviews (1): Last reviewed commit: "fix(openai): mint workload identity toke..." | Re-trigger Greptile}

[mateo-berri]

bugbot run

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 19da217. Configure here.}

[tin-berri]

LGTM