Skip to content
162 changes: 162 additions & 0 deletions docs/providers/anthropic.md
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,168 @@ print(response)
**Finding your Azure endpoint:** Go to Azure AI Foundry → Your deployment → Overview. Your base URL will be `https://<resource-name>.services.ai.azure.com/anthropic`
:::

## Workload Identity Federation

Anthropic supports workload identity federation, so a proxy can exchange an OIDC identity token it already holds for a short-lived `sk-ant-oat01` access token instead of storing a long-lived `sk-ant-` key. This is the Anthropic equivalent of what Vertex AI and Azure already do, and it suits deployments with a no-static-secrets policy.

LiteLLM mints and caches the access token for you. Configure the federation identifiers plus one identity source, and every route that reaches Anthropic uses the minted token: chat completions, `/v1/messages`, files, batches, skills, passthrough, token counting and model discovery.

### Federation identifiers

These come from the federation rule you create in the Anthropic Console, and they are the same whichever identity source you pick:

| Field | Description |
| --- | --- |
| `anthropic_federation_rule_id` | The `fdrl_...` id of the federation rule |
| `anthropic_organization_id` | Your Anthropic organization UUID |
| `anthropic_service_account_id` | The `svac_...` service account the rule maps to |
| `anthropic_federation_workspace_id` | Required when the rule is enabled in more than one workspace; scopes the minted token to that workspace |

Anthropic's own reference calls the last one `workspace_id`, but federation reads it from `ANTHROPIC_FEDERATION_WORKSPACE_ID`, not `ANTHROPIC_WORKSPACE_ID`. The shorter name already belongs to the Bedrock Claude platform provider, where it picks the workspace a Bedrock call is billed to, so federation takes the longer one and leaves that behavior alone. Setting `ANTHROPIC_WORKSPACE_ID` does nothing for federation

### Static keys take precedence

A static credential outranks federation everywhere in the Anthropic provider. If `ANTHROPIC_API_KEY` is set, every Anthropic route uses it and nothing is federated; `ANTHROPIC_AUTH_TOKEN` comes next, and federation is the last tier. This matches the Anthropic SDK's own ordering, and it applies to files, batches and model discovery as well as to chat.

So a deployment that is meant to be federated must not carry a static key. If one is set anyway, LiteLLM logs a warning naming the model whose federation is being shadowed. A blank or whitespace-only value counts as unset and falls through to federation

### Configuring by environment instead

Every field below can come from the environment rather than the deployment, which is what you want
when the same values apply proxy-wide. A value set on the deployment wins over the environment.

| Field | Environment variable |
| --- | --- |
| `anthropic_federation_rule_id` | `ANTHROPIC_FEDERATION_RULE_ID` |
| `anthropic_organization_id` | `ANTHROPIC_ORGANIZATION_ID` |
| `anthropic_service_account_id` | `ANTHROPIC_SERVICE_ACCOUNT_ID` |
| `anthropic_federation_workspace_id` | `ANTHROPIC_FEDERATION_WORKSPACE_ID` |
| `anthropic_identity_source` | `ANTHROPIC_IDENTITY_SOURCE` |
| `anthropic_identity_token_file` | `ANTHROPIC_IDENTITY_TOKEN_FILE` |
| `anthropic_identity_token` | `ANTHROPIC_IDENTITY_TOKEN` |

### Identity sources

There are four ways to supply the OIDC assertion. Two need no `anthropic_identity_source` at all, and two are selected with it.

#### Token file (default)

Reads an assertion a platform already projects onto disk, which is how Kubernetes service account tokens and most CI runners work. Set `anthropic_identity_token_file` and leave `anthropic_identity_source` unset. The path must sit under LiteLLM's OIDC file allowlist, which you extend with `LITELLM_OIDC_ALLOWED_CREDENTIAL_DIRS`. The file is re-read on each mint, so a rotated token is picked up without a restart.
Comment thread
derhornspieler marked this conversation as resolved.

```yaml
model_list:
- model_name: claude-sonnet-4-5
litellm_params:
model: anthropic/claude-sonnet-4-5
anthropic_identity_token_file: /var/run/secrets/anthropic.com/token
anthropic_federation_rule_id: os.environ/ANTHROPIC_FEDERATION_RULE_ID
anthropic_organization_id: os.environ/ANTHROPIC_ORGANIZATION_ID
anthropic_service_account_id: os.environ/ANTHROPIC_SERVICE_ACCOUNT_ID
```

#### Secret reference (default)

`anthropic_identity_token` takes an `oidc/` reference, not a raw token. Accepted forms are `oidc/env/VAR_NAME`, `oidc/file//absolute/path`, `oidc/github/<audience>`, and `oidc/google/<audience>`. Pasting a bare JWT is rejected, so export it and reference the variable instead.

```yaml
anthropic_identity_token: oidc/env/MY_WORKLOAD_TOKEN
```

#### LiteLLM as the issuer

For deployments with no external IdP, LiteLLM signs the assertion itself with an ES256 (P-256) key. Set `anthropic_identity_source: internal_issuer` and point `anthropic_issuer_signing_key_ref` at the key rather than pasting it inline, so custody stays with your secret manager. `anthropic_issuer_ttl_seconds` defaults to 300 and cannot exceed 3600.

```yaml
credential_list:
- credential_name: anthropic_wif
credential_values:
anthropic_identity_source: internal_issuer
anthropic_issuer_url: https://litellm.example.com
anthropic_issuer_subject: litellm-proxy
anthropic_issuer_audience: https://api.anthropic.com
anthropic_issuer_signing_key_ref: os.environ/ANTHROPIC_WIF_SIGNING_KEY
anthropic_issuer_ttl_seconds: 300
anthropic_federation_rule_id: os.environ/ANTHROPIC_FEDERATION_RULE_ID
anthropic_organization_id: os.environ/ANTHROPIC_ORGANIZATION_ID
anthropic_service_account_id: os.environ/ANTHROPIC_SERVICE_ACCOUNT_ID
credential_info:
custom_llm_provider: anthropic

model_list:
- model_name: claude-sonnet-4-5
litellm_params:
model: anthropic/claude-sonnet-4-5
litellm_credential_name: anthropic_wif
```

Anthropic needs the matching public key to verify what LiteLLM signs. Export it from the proxy and register it as the **inline** issuer JWKS on your federation rule:

```shell
curl -s http://localhost:4000/credentials/anthropic_wif/jwks \
-H "Authorization: Bearer $LITELLM_MASTER_KEY"
```

The endpoint is proxy-admin only and never exposes the private key, so paste the JSON it returns into the Anthropic Console rather than pointing Anthropic at the URL. A freshly registered JWKS takes about a minute before Anthropic accepts assertions signed by it.

#### Keycloak

For shops that already run Keycloak as the workload IdP, LiteLLM fetches the assertion with an OAuth client credentials grant. Set `anthropic_identity_source: keycloak` plus:

```yaml
anthropic_keycloak_token_url: https://keycloak.example.com/realms/prod/protocol/openid-connect/token
anthropic_keycloak_client_id: litellm-proxy
anthropic_keycloak_auth_method: client_secret_basic # or client_secret_post
anthropic_keycloak_client_secret_ref: os.environ/KEYCLOAK_CLIENT_SECRET
anthropic_keycloak_scope: anthropic-federation
```

Mixing fields across variants is rejected rather than silently ignored, so a config that names `internal_issuer` while carrying Keycloak fields fails at startup instead of quietly falling back.

### Setting it up in the UI

The Admin UI does not offer these fields yet, so configure federation through the proxy config or the environment as shown above. A follow-up adds them to the LLM Credentials flow.

### Token lifetime and refresh

LiteLLM caches the minted access token per deployment and refreshes it in the background before it expires, so a request rarely waits on an exchange. Refresh is two-tier: an advisory refresh at half the token's lifetime that happens in the background while the old token keeps serving, and a mandatory one at an eighth of the lifetime that blocks. Concurrent requests for the same deployment share a single in-flight exchange rather than each minting their own, so the number of exchanges does not scale with traffic.

Workers on the same host share the minted token through a small on-disk cache under the temp directory, readable only by the proxy's user, so a proxy running several uvicorn workers exchanges each assertion once rather than once per worker. `LITELLM_TOKEN_EXCHANGE_CACHE_DIR` moves that cache, and setting it to an empty string turns it off so every process mints on its own. Replicas on different hosts always mint their own token.

Anthropic accepts each assertion once: a second exchange of the same JWT is denied with the opaque 401 and shows up as `jti_reused` in the rule's authentication history. That is fine for the internal issuer and Keycloak sources, which mint a fresh assertion for every exchange, but a token file or `oidc/env/` value has to rotate faster than the rule's `token_lifetime_seconds`, or the first refresh after the minted token expires fails until a new assertion shows up. Kubernetes rotates a projected service account token once 80% of its `expirationSeconds` has passed, so keep the rule's token lifetime at or above that rotation period; the defaults (a one hour projected token and a one hour rule lifetime) line up.

### Who can configure it

Only a proxy admin can create or change a deployment that uses workload identity federation. A team admin who otherwise manages a team-scoped deployment gets a 403, and that holds however the change is expressed: setting a federation field directly, attaching a credential that carries one by name, or changing `api_base` on a deployment that is already federated. The last one matters because `api_base` decides where the signed assertion is sent and where the minted token is presented, so it is part of the federation configuration even though it is not a federation field.

The same rule covers the stored fallback lists on a key or team. A fallback target cannot carry a federation field, since those are merged over the deployment's own configuration when a failover happens.

Teams keep normal access to a federated deployment. Only editing it moves to the proxy admin.

### Restricting where assertions are sent

The exchange only talks to `api.anthropic.com`. If you front Anthropic with a gateway, list its hostname in `LITELLM_ANTHROPIC_WIF_ALLOWED_HOSTS` (comma separated) so the signed assertion is allowed to reach it.

```shell
export LITELLM_ANTHROPIC_WIF_ALLOWED_HOSTS="anthropic.gateway.internal"
```

An entry may name a port, in which case only that port is trusted and another process on the same host is not. An entry without a port trusts every port on that host.

```shell
export LITELLM_ANTHROPIC_WIF_ALLOWED_HOSTS="anthropic.gateway.internal:8443"
```

The list is read from the environment only. It is never taken from a model or credential API, because `api_base` decides both where the assertion is sent and where the minted token is presented

### Monitoring

Token health is emitted through the standard service-logging path: `prometheus_system` sends it to Prometheus and `otel` sends it to OpenTelemetry, using the exporter settings from the [OpenTelemetry docs](../observability/opentelemetry_integration). The services are `anthropic_wif` for the exchange itself and `anthropic_wif_cache` for cache hits and misses, giving you mint counts, mint latency, and failures broken out by cause. Enable them with:

```yaml
litellm_settings:
service_callback: ["prometheus_system", "otel"]
```

## Usage

```python
Expand Down