From 6d6119ada86c348d054ba964ff11c74630ee233c Mon Sep 17 00:00:00 2001 From: jesus Date: Thu, 8 Oct 2026 01:41:48 +0000 Subject: [PATCH 1/3] docs: correct Claude subscription and ChatGPT subscription credential behavior Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --- docs/providers/chatgpt.md | 16 +++- .../tutorials/claude_code_max_subscription.md | 91 ++++++++++++------- 2 files changed, 74 insertions(+), 33 deletions(-) diff --git a/docs/providers/chatgpt.md b/docs/providers/chatgpt.md index a5dc7a307..041c90196 100644 --- a/docs/providers/chatgpt.md +++ b/docs/providers/chatgpt.md @@ -1,6 +1,6 @@ # ChatGPT Subscription -Use ChatGPT Pro/Max subscription models through LiteLLM with OAuth device flow authentication. +Use a ChatGPT subscription that includes Codex (see OpenAI's [Codex pricing](https://developers.openai.com/codex/pricing) for the plans, from Plus and Pro to Business and Enterprise) through LiteLLM with OAuth device flow authentication. | Property | Details | |-------|-------| @@ -19,6 +19,8 @@ Notes: ChatGPT subscription access uses an OAuth device code flow. The LiteLLM Python SDK runs it the first time it needs a token, provided it is called from a synchronous script on the main thread: it prints a verification URL and a device code, you open the URL, sign in, and enter the code, and the tokens are stored in `~/.config/litellm/chatgpt/auth.json` for reuse. Async code, notebooks, worker threads, and the LiteLLM proxy cannot answer that prompt (older releases block on it, newer ones fail with `ChatGPT device-code login needs a human`), so for the proxy you sign in first and mount the resulting file, as described in [Sign in before starting the proxy](#sign-in-before-starting-the-proxy). +Device code login has to be allowed for the account first: in ChatGPT's security settings for a personal plan, or by a workspace admin in the workspace permissions for Business and Enterprise ([OpenAI docs](https://learn.chatgpt.com/docs/auth)). + ## Usage - LiteLLM Python SDK ### Responses (recommended for Codex models) @@ -151,6 +153,18 @@ model_list: litellm --config config.yaml ``` +## Credential Scope and Attribution + +A proxy uses exactly one ChatGPT login: the `auth.json` in `CHATGPT_TOKEN_DIR`, which is set once for the whole process. Every request to a `chatgpt/` model goes upstream with that file's access token and the `ChatGPT-Account-Id` taken from it, whatever virtual key made the call, and LiteLLM ignores any ChatGPT token a client sends. Every key, user, and team that can call a `chatgpt/` model therefore uses the same ChatGPT account and workspace, and users cannot bring their own ChatGPT subscription through this provider. Serving several ChatGPT accounts takes one proxy deployment per account. + +LiteLLM attributes `chatgpt/` requests to the calling virtual key, user, and team like any other model and records their token counts. The models have no price in the cost map, so recorded spend is $0 and `max_budget` limits never trigger; cap usage with `rpm_limit` or `tpm_limit` on keys and teams instead. On the OpenAI side, all of the usage shows up under the account that signed in. + +## Refresh Failures and Revocation + +LiteLLM refreshes only when the stored access token is expired. A 401 from the ChatGPT backend for a token that has not expired yet, for example after the session was signed out on OpenAI's side, is returned to the caller and does not trigger a refresh. When the refresh token is rejected, the proxy logs `ChatGPT refresh token failed, re-login required` and requests fail with HTTP 400 `ChatGPT device-code login needs a human`; sign in again outside the proxy and replace `auth.json`. + +LiteLLM has no logout or revoke call for this provider. To stop the proxy using the account, delete `auth.json` (and the Secret it was seeded from on Kubernetes) and restart the proxy. To invalidate the tokens themselves, sign the session out from the ChatGPT account's security settings. + ## Configuration ### Environment Variables diff --git a/docs/tutorials/claude_code_max_subscription.md b/docs/tutorials/claude_code_max_subscription.md index f94173138..2b52c9e3b 100644 --- a/docs/tutorials/claude_code_max_subscription.md +++ b/docs/tutorials/claude_code_max_subscription.md @@ -2,21 +2,21 @@ import Image from '@theme/IdealImage'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; -# Using Claude Code Max Subscription +# Using Claude Code with a Claude Subscription (Pro, Max, Team, Enterprise)
-Route Claude Code Max subscription traffic through LiteLLM AI Gateway. +Route Claude Code traffic signed in with a Claude Pro, Max, Team, or Enterprise subscription through LiteLLM AI Gateway.
-**Why Claude Code Max over direct API?** -- **Lower costs** — Claude Code Max subscriptions are cheaper for Claude Code power users than per-token API pricing +**Why a Claude subscription over direct API?** +- **Lower costs**: Claude subscriptions are cheaper for Claude Code power users than per-token API pricing **Why route through LiteLLM?** -- **Cost attribution** — Track spend per user, team, or key -- **Budgets & rate limits** — Set spending caps and request limits -- **Guardrails** — Apply content filtering and safety controls to all requests +- **Cost attribution**: Track spend per user, team, or key +- **Budgets & rate limits**: Set spending caps and request limits +- **Guardrails**: Apply content filtering and safety controls to all requests @@ -29,12 +29,12 @@ Watch the end-to-end walkthrough of setting up Claude Code with LiteLLM Gateway: ## Prerequisites - [Claude Code](https://docs.anthropic.com/en/docs/claude-code/overview) installed -- Claude Max subscription -- LiteLLM Gateway running +- A Claude Pro, Max, Team, or Enterprise subscription (Team and Enterprise members sign in with the Claude account their admin invited) +- LiteLLM Gateway v1.82.3 or later running ## Step 1: Configure LiteLLM Proxy -Create a `config.yaml` with the critical `forward_client_headers_to_llm_api: true` setting: +Create a `config.yaml` with your Anthropic models: ```yaml showLineNumbers title="config.yaml" model_list: @@ -52,12 +52,13 @@ model_list: general_settings: master_key: os.environ/LITELLM_MASTER_KEY - forward_client_headers_to_llm_api: true # Required: forwards OAuth token to Anthropic ``` -:::info[Why `forward_client_headers_to_llm_api`?] +:::info[No header forwarding setting is needed] -This setting forwards the user's OAuth token (in the `Authorization` header) through LiteLLM to the Anthropic API, enabling per-user authentication with their Max subscription while LiteLLM handles tracking and controls. +Since v1.82.3, LiteLLM forwards a client's `Authorization: Bearer sk-ant-oat...` subscription token to `anthropic/` deployments without `forward_client_headers_to_llm_api`. The token takes precedence over any `api_key` set on the deployment, and LiteLLM adds the `anthropic-beta: oauth-2025-04-20` header Anthropic requires for OAuth. Turn on `forward_client_headers_to_llm_api` only if you also want other client headers forwarded. + +A deployment without an `api_key` serves only subscription users: a request that arrives without a subscription token fails with `401 Missing Anthropic API Key`. If you set an `api_key`, requests without a token fall back to it and are billed to that API key. ::: @@ -109,9 +110,9 @@ Click "Create Key" to generate your virtual key. Copy the generated key value (e --- -### Part 2: Sign into Claude Code Max Plan (Client Side) +### Part 2: Sign into Claude Code with Your Subscription (Client Side) -Set up Claude Code environment variables and authenticate with your Max subscription. +Set up Claude Code environment variables and authenticate with your Claude subscription. #### 2.1 Set Environment Variables @@ -133,6 +134,8 @@ export ANTHROPIC_CUSTOM_HEADERS="x-litellm-api-key: Bearer sk-otsclFlEblQ-6D60ua | `ANTHROPIC_MODEL` | The model name configured in your LiteLLM `config.yaml` | | `ANTHROPIC_CUSTOM_HEADERS` | The `x-litellm-api-key` header for LiteLLM authentication | +Do not also set `ANTHROPIC_AUTH_TOKEN`, `ANTHROPIC_API_KEY`, or an `apiKeyHelper`. Claude Code sends any of those in place of the subscription login, so the request no longer uses the subscription. + #### 2.2 Launch Claude Code Start Claude Code: @@ -151,7 +154,7 @@ Choose "Claude account with subscription" (Pro, Max, Team, or Enterprise). #### 2.4 Authorize in Browser -Claude Code opens your browser to authenticate. Click "Authorize" to connect your Claude Max account. +Claude Code opens your browser to authenticate. Click "Authorize" to connect your Claude account. @@ -206,7 +209,7 @@ The logs show: LiteLLM Gateway handles two types of authentication: 1. **`x-litellm-api-key`**: Authenticates the request with LiteLLM (usage tracking, budgets, rate limits) -2. **OAuth Token (via `Authorization` header)**: Forwarded to Anthropic API for Claude Max authentication +2. **OAuth Token (via `Authorization` header)**: Forwarded to Anthropic API for Claude subscription authentication ```mermaid sequenceDiagram @@ -218,9 +221,9 @@ sequenceDiagram Note over LiteLLM: 1. Validate x-litellm-api-key
2. Check budgets/rate limits
3. Log request for tracking - LiteLLM->>Anthropic: Forward request with:
- Authorization: Bearer {oauth_token}
(User's Claude Max OAuth token) + LiteLLM->>Anthropic: Forward request with:
- Authorization: Bearer {oauth_token}
(User's Claude subscription OAuth token) - Note over Anthropic: Authenticate user via
OAuth token from Max plan + Note over Anthropic: Authenticate user via
OAuth token from their plan Anthropic-->>LiteLLM: Response @@ -234,7 +237,7 @@ sequenceDiagram | Header | Purpose | Handled By | |--------|---------|------------| | `x-litellm-api-key` | LiteLLM Gateway authentication, budget tracking, rate limits | LiteLLM | -| `Authorization: Bearer {oauth_token}` | Claude Max subscription authentication | Anthropic API | +| `Authorization: Bearer {oauth_token}` | Claude subscription authentication | Anthropic API | ### Complete Request Flow Example @@ -243,7 +246,7 @@ Here's what a typical request looks like when Claude Code makes a call through L ```bash showLineNumbers title="Example Request from Claude Code to LiteLLM" curl -X POST "http://localhost:4000/v1/messages" \ -H "x-litellm-api-key: Bearer sk-otsclFlEblQ-6D60ua2IZg" \ - -H "Authorization: Bearer oauth_token_from_max_plan" \ + -H "Authorization: Bearer sk-ant-oat01-..." \ -H "Content-Type: application/json" \ -d '{ "model": "anthropic-claude", @@ -255,13 +258,43 @@ curl -X POST "http://localhost:4000/v1/messages" \ LiteLLM then: 1. Validates `x-litellm-api-key` for gateway access 2. Logs the request for usage tracking -3. Forwards the request to Anthropic with the OAuth `Authorization` header (because of `forward_client_headers_to_llm_api: true`) +3. Forwards the request to Anthropic with the OAuth `Authorization` header in place of any configured `x-api-key` + +## Plans, Credentials, and Attribution + +### Supported plans + +LiteLLM does not check which plan issued the token. Pro, Max, Team, and Enterprise logins all give Claude Code an `sk-ant-oat` OAuth token, and LiteLLM handles every one of them the same way. Plan limits, usage reporting, and billing are applied by Anthropic to the Claude account that signed in, and Team and Enterprise admin controls such as seat assignment, SSO, and member removal stay in Claude's admin console. To keep members on your organization instead of a personal account, set Claude Code's `forceLoginOrgUUID` [setting](https://code.claude.com/docs/en/authentication#restrict-login-to-your-organization). + +Anthropic's [Legal and compliance](https://code.claude.com/docs/en/legal-and-compliance) page restricts using subscription credentials on behalf of other users. Each developer signs in with their own account and LiteLLM passes that token through per request; do not put a subscription token in a deployment's `api_key` to share it. + +### Which requests use the subscription token + +The token is forwarded on `/v1/messages` and on `/v1/chat/completions` when the model is an `anthropic/` deployment. It is never sent to other providers, including Claude on Bedrock or Vertex AI, which keep using their own credentials (since v1.99.0). `/v1/messages/count_tokens` uses the deployment's configured key, not the subscription token. The `/anthropic` pass-through route does not use the subscription token when the proxy has an Anthropic key configured, so point `ANTHROPIC_BASE_URL` at the proxy root as shown above rather than at `/anthropic`. + +### Credential storage, refresh, and revocation + +Claude Code owns the subscription credential. It stores the login locally (the macOS Keychain, or `~/.claude/.credentials.json` on Linux and Windows, under `CLAUDE_CONFIG_DIR` when that is set), refreshes it on its own, and `/logout` signs it out. LiteLLM stores nothing: the token is used for the one upstream request it arrived with, it is not cached across requests, and it is not written to logs or spend logs. Concurrent requests from different users each go upstream with their own token. + +When a token is expired or revoked, Anthropic answers `401 OAuth access token is invalid.` and LiteLLM returns that to Claude Code; the user runs `/login` again. LiteLLM cannot revoke a Claude login. To cut a user off at the gateway, [block](/docs/proxy/virtual_keys) or delete their virtual key, which stops their requests through LiteLLM; their Claude login itself still works directly against Anthropic until they or their admin remove it. + +One user's expired or revoked token can currently block everyone else on the same model. LiteLLM counts the upstream 401 as a deployment failure and puts the deployment into cooldown, and while the cooldown lasts every other user's request to that model fails with `429 No deployments available for selected model` without reaching Anthropic. On proxies that serve subscription users, turn cooldowns off: + +```yaml showLineNumbers title="config.yaml - Keep one user's 401 from blocking others" +router_settings: + disable_cooldowns: true +``` +### Attribution and cost + +Spend is attributed to the virtual key in `x-litellm-api-key` and the user and team it belongs to, so per-user and per-team reporting, budgets, and rate limits work as usual. LiteLLM does not see which Claude account or organization the token belongs to. Cost is calculated at Anthropic API list prices and counts toward key, user, and team budgets even though Anthropic bills the usage to the subscription, so treat budgets here as usage caps in API-equivalent dollars. + +Starting with v1.105.0, each spend log records `used_client_oauth_token: true` when the request went upstream with the client's subscription token and `false` when it used the configured key. The Logs page shows this as the Credential column ("Client OAuth token" or "Configured key") and can filter on it. Requests through the `/anthropic` pass-through route do not carry the flag. ## Advanced Configuration ### Per-Model Header Forwarding -For more granular control, you can enable header forwarding only for specific models: +The subscription token is forwarded without this setting. If you also want other client headers forwarded, you can enable header forwarding only for specific models: ```yaml showLineNumbers title="config.yaml - Per-Model Header Forwarding" model_list: @@ -285,7 +318,7 @@ litellm_settings: ### Budget Controls -Set up per-user budgets while using Max subscriptions: +Set up per-user budgets while using Claude subscriptions (cost is tracked at API list prices, see [Attribution and cost](#attribution-and-cost)): ```yaml showLineNumbers title="config.yaml - With Database for Budget Tracking" model_list: @@ -295,7 +328,6 @@ model_list: general_settings: master_key: os.environ/LITELLM_MASTER_KEY - forward_client_headers_to_llm_api: true database_url: "postgresql://..." ``` @@ -316,14 +348,9 @@ curl -X POST "http://localhost:4000/key/generate" \ ### OAuth Token Not Being Forwarded -**Symptom**: Authentication errors from Anthropic API - -**Solution**: Ensure `forward_client_headers_to_llm_api: true` is set in your config: +**Symptom**: Authentication errors from Anthropic API, or usage billed to the configured API key instead of the subscription -```yaml showLineNumbers title="config.yaml - Enable Header Forwarding" -general_settings: - forward_client_headers_to_llm_api: true -``` +**Solution**: Check that you are on LiteLLM v1.82.3 or later, that the model is an `anthropic/` deployment, and that Claude Code reaches the proxy root (`/v1/messages`) rather than `/anthropic`. In Claude Code, `/status` shows the active login; if `ANTHROPIC_AUTH_TOKEN`, `ANTHROPIC_API_KEY`, or an `apiKeyHelper` is set, Claude Code sends that instead of the subscription token. A `401 OAuth access token is invalid.` means the login expired or was revoked; run `/login`. ### LiteLLM Authentication Failing From 2421d057c4c0c39bc2dc910f98a6e9bda7c23cb7 Mon Sep 17 00:00:00 2001 From: jesus Date: Fri, 9 Oct 2026 22:03:48 +0000 Subject: [PATCH 2/3] docs: fold OAuth spend-log flag paragraph into the seat billing section Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --- docs/tutorials/claude_code_max_subscription.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/tutorials/claude_code_max_subscription.md b/docs/tutorials/claude_code_max_subscription.md index 83c39aaa8..7db822ac7 100644 --- a/docs/tutorials/claude_code_max_subscription.md +++ b/docs/tutorials/claude_code_max_subscription.md @@ -274,7 +274,7 @@ The token is forwarded on `/v1/messages` and on `/v1/chat/completions` when the ### Credential storage, refresh, and revocation -Claude Code owns the subscription credential. It stores the login locally (the macOS Keychain, or `~/.claude/.credentials.json` on Linux and Windows, under `CLAUDE_CONFIG_DIR` when that is set), refreshes it on its own, and `/logout` signs it out. LiteLLM stores nothing: the token is used for the one upstream request it arrived with, it is not cached across requests, and it is not written to logs or spend logs. Concurrent requests from different users each go upstream with their own token. +Claude Code owns the subscription credential. It stores the login locally (the macOS Keychain, or `~/.claude/.credentials.json` on Linux and Windows, under `CLAUDE_CONFIG_DIR` when that is set), refreshes it on its own, and `/logout` signs it out. LiteLLM stores nothing: the token is used for the one upstream request it arrived with, it is not cached across requests, and it is not written to logs or spend logs (spend logs record only whether one was used, see [Seeing Which Requests Were Billed to a Seat](#seeing-which-requests-were-billed-to-a-seat)). Concurrent requests from different users each go upstream with their own token. When a token is expired or revoked, Anthropic answers `401 OAuth access token is invalid.` and LiteLLM returns that to Claude Code; the user runs `/login` again. LiteLLM cannot revoke a Claude login. To cut a user off at the gateway, [block](/docs/proxy/virtual_keys) or delete their virtual key, which stops their requests through LiteLLM; their Claude login itself still works directly against Anthropic until they or their admin remove it. @@ -284,11 +284,12 @@ One user's expired or revoked token can currently block everyone else on the sam router_settings: disable_cooldowns: true ``` + ### Attribution and cost Spend is attributed to the virtual key in `x-litellm-api-key` and the user and team it belongs to, so per-user and per-team reporting, budgets, and rate limits work as usual. LiteLLM does not see which Claude account or organization the token belongs to. Cost is calculated at Anthropic API list prices and counts toward key, user, and team budgets even though Anthropic bills the usage to the subscription, so treat budgets here as usage caps in API-equivalent dollars. -Starting with v1.105.0, each spend log records `used_client_oauth_token: true` when the request went upstream with the client's subscription token and `false` when it used the configured key. The Logs page shows this as the Credential column ("Client OAuth token" or "Configured key") and can filter on it. Requests through the `/anthropic` pass-through route do not carry the flag. +To tell subscription-billed requests from key-billed ones, see [Seeing Which Requests Were Billed to a Seat](#seeing-which-requests-were-billed-to-a-seat). ## Advanced Configuration @@ -346,7 +347,7 @@ curl -X POST "http://localhost:4000/key/generate" \ ### Seeing Which Requests Were Billed to a Seat -Every spend log row records which credential the upstream call used in `metadata.used_client_oauth_token`: `true` when the request went to Anthropic with the developer's forwarded OAuth token (the Max seat paid for it), `false` when it went out with the deployment's configured `api_key`. The token itself is never written to the log. The field needs LiteLLM v1.105.0 or later (first in `v1.105.0-rc.1`), and rows written by an earlier version have no value, so they match neither filter below. A request the router sends to a Bedrock or Vertex deployment reads `false` even when the client sent an OAuth token, since only the direct Anthropic route forwards it +Every spend log row records which credential the upstream call used in `metadata.used_client_oauth_token`: `true` when the request went to Anthropic with the developer's forwarded OAuth token (the subscription seat paid for it), `false` when it went out with the deployment's configured `api_key`. The token itself is never written to the log. The field needs LiteLLM v1.105.0 or later (first in `v1.105.0-rc.1`), and rows written by an earlier version have no value, so they match neither filter below. A request the router sends to a Bedrock or Vertex deployment reads `false` even when the client sent an OAuth token, since only the direct Anthropic route forwards it. Requests through the `/anthropic` pass-through route do not carry the field `spend` stays at the model's list price on both kinds of rows, so budgets and rate limits keep working across seat-billed and key-billed traffic. To get the real API bill, subtract the seat-billed rows From 2a4af9d1fcd2874cf7abf7c9fb7354933fb35651 Mon Sep 17 00:00:00 2001 From: jesus Date: Sat, 10 Oct 2026 03:26:31 +0000 Subject: [PATCH 3/3] docs: OAuth token forwarding without the setting shipped in v1.81.14 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --- docs/tutorials/claude_code_max_subscription.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/tutorials/claude_code_max_subscription.md b/docs/tutorials/claude_code_max_subscription.md index 7db822ac7..bffa1b471 100644 --- a/docs/tutorials/claude_code_max_subscription.md +++ b/docs/tutorials/claude_code_max_subscription.md @@ -30,7 +30,7 @@ Watch the end-to-end walkthrough of setting up Claude Code with LiteLLM Gateway: - [Claude Code](https://docs.anthropic.com/en/docs/claude-code/overview) installed - A Claude Pro, Max, Team, or Enterprise subscription (Team and Enterprise members sign in with the Claude account their admin invited) -- LiteLLM Gateway v1.82.3 or later running +- LiteLLM Gateway v1.81.14 or later running ## Step 1: Configure LiteLLM Proxy @@ -56,7 +56,7 @@ general_settings: :::info[No header forwarding setting is needed] -Since v1.82.3, LiteLLM forwards a client's `Authorization: Bearer sk-ant-oat...` subscription token to `anthropic/` deployments without `forward_client_headers_to_llm_api`. The token takes precedence over any `api_key` set on the deployment, and LiteLLM adds the `anthropic-beta: oauth-2025-04-20` header Anthropic requires for OAuth. Turn on `forward_client_headers_to_llm_api` only if you also want other client headers forwarded. +Since v1.81.14, LiteLLM forwards a client's `Authorization: Bearer sk-ant-oat...` subscription token to `anthropic/` deployments without `forward_client_headers_to_llm_api`. The token takes precedence over any `api_key` set on the deployment, and LiteLLM adds the `anthropic-beta: oauth-2025-04-20` header Anthropic requires for OAuth. Turn on `forward_client_headers_to_llm_api` only if you also want other client headers forwarded. A deployment without an `api_key` serves only subscription users: a request that arrives without a subscription token fails with `401 Missing Anthropic API Key`. If you set an `api_key`, requests without a token fall back to it and are billed to that API key. @@ -366,7 +366,7 @@ Pass `used_client_oauth_token=false` for the requests the configured key paid fo **Symptom**: Authentication errors from Anthropic API, or usage billed to the configured API key instead of the subscription -**Solution**: Check that you are on LiteLLM v1.82.3 or later, that the model is an `anthropic/` deployment, and that Claude Code reaches the proxy root (`/v1/messages`) rather than `/anthropic`. In Claude Code, `/status` shows the active login; if `ANTHROPIC_AUTH_TOKEN`, `ANTHROPIC_API_KEY`, or an `apiKeyHelper` is set, Claude Code sends that instead of the subscription token. A `401 OAuth access token is invalid.` means the login expired or was revoked; run `/login`. +**Solution**: Check that you are on LiteLLM v1.81.14 or later, that the model is an `anthropic/` deployment, and that Claude Code reaches the proxy root (`/v1/messages`) rather than `/anthropic`. In Claude Code, `/status` shows the active login; if `ANTHROPIC_AUTH_TOKEN`, `ANTHROPIC_API_KEY`, or an `apiKeyHelper` is set, Claude Code sends that instead of the subscription token. A `401 OAuth access token is invalid.` means the login expired or was revoked; run `/login`. ### LiteLLM Authentication Failing