diff --git a/docs/providers/anthropic.md b/docs/providers/anthropic.md index e1333acff..f7467d7e2 100644 --- a/docs/providers/anthropic.md +++ b/docs/providers/anthropic.md @@ -227,6 +227,168 @@ print(response) **Finding your Azure endpoint:** Go to Azure AI Foundry → Your deployment → Overview. Your base URL will be `https://.services.ai.azure.com/anthropic` ::: +## Workload Identity Federation + +Anthropic supports workload identity federation, so a proxy can exchange an OIDC identity token it already holds for a short-lived `sk-ant-oat01` access token instead of storing a long-lived `sk-ant-` key. This is the Anthropic equivalent of what Vertex AI and Azure already do, and it suits deployments with a no-static-secrets policy. + +LiteLLM mints and caches the access token for you. Configure the federation identifiers plus one identity source, and every route that reaches Anthropic uses the minted token: chat completions, `/v1/messages`, files, batches, skills, passthrough, token counting and model discovery. + +### Federation identifiers + +These come from the federation rule you create in the Anthropic Console, and they are the same whichever identity source you pick: + +| Field | Description | +| --- | --- | +| `anthropic_federation_rule_id` | The `fdrl_...` id of the federation rule | +| `anthropic_organization_id` | Your Anthropic organization UUID | +| `anthropic_service_account_id` | The `svac_...` service account the rule maps to | +| `anthropic_federation_workspace_id` | Required when the rule is enabled in more than one workspace; scopes the minted token to that workspace | + +Anthropic's own reference calls the last one `workspace_id`, but federation reads it from `ANTHROPIC_FEDERATION_WORKSPACE_ID`, not `ANTHROPIC_WORKSPACE_ID`. The shorter name already belongs to the Bedrock Claude platform provider, where it picks the workspace a Bedrock call is billed to, so federation takes the longer one and leaves that behavior alone. Setting `ANTHROPIC_WORKSPACE_ID` does nothing for federation + +### Static keys take precedence + +A static credential outranks federation everywhere in the Anthropic provider. If `ANTHROPIC_API_KEY` is set, every Anthropic route uses it and nothing is federated; `ANTHROPIC_AUTH_TOKEN` comes next, and federation is the last tier. This matches the Anthropic SDK's own ordering, and it applies to files, batches and model discovery as well as to chat. + +So a deployment that is meant to be federated must not carry a static key. If one is set anyway, LiteLLM logs a warning naming the model whose federation is being shadowed. A blank or whitespace-only value counts as unset and falls through to federation + +### Configuring by environment instead + +Every field below can come from the environment rather than the deployment, which is what you want +when the same values apply proxy-wide. A value set on the deployment wins over the environment. + +| Field | Environment variable | +| --- | --- | +| `anthropic_federation_rule_id` | `ANTHROPIC_FEDERATION_RULE_ID` | +| `anthropic_organization_id` | `ANTHROPIC_ORGANIZATION_ID` | +| `anthropic_service_account_id` | `ANTHROPIC_SERVICE_ACCOUNT_ID` | +| `anthropic_federation_workspace_id` | `ANTHROPIC_FEDERATION_WORKSPACE_ID` | +| `anthropic_identity_source` | `ANTHROPIC_IDENTITY_SOURCE` | +| `anthropic_identity_token_file` | `ANTHROPIC_IDENTITY_TOKEN_FILE` | +| `anthropic_identity_token` | `ANTHROPIC_IDENTITY_TOKEN` | + +### Identity sources + +There are four ways to supply the OIDC assertion. Two need no `anthropic_identity_source` at all, and two are selected with it. + +#### Token file (default) + +Reads an assertion a platform already projects onto disk, which is how Kubernetes service account tokens and most CI runners work. Set `anthropic_identity_token_file` and leave `anthropic_identity_source` unset. The path must sit under LiteLLM's OIDC file allowlist, which you extend with `LITELLM_OIDC_ALLOWED_CREDENTIAL_DIRS`. The file is re-read on each mint, so a rotated token is picked up without a restart. + +```yaml +model_list: + - model_name: claude-sonnet-4-5 + litellm_params: + model: anthropic/claude-sonnet-4-5 + anthropic_identity_token_file: /var/run/secrets/anthropic.com/token + anthropic_federation_rule_id: os.environ/ANTHROPIC_FEDERATION_RULE_ID + anthropic_organization_id: os.environ/ANTHROPIC_ORGANIZATION_ID + anthropic_service_account_id: os.environ/ANTHROPIC_SERVICE_ACCOUNT_ID +``` + +#### Secret reference (default) + +`anthropic_identity_token` takes an `oidc/` reference, not a raw token. Accepted forms are `oidc/env/VAR_NAME`, `oidc/file//absolute/path`, `oidc/github/`, and `oidc/google/`. Pasting a bare JWT is rejected, so export it and reference the variable instead. + +```yaml + anthropic_identity_token: oidc/env/MY_WORKLOAD_TOKEN +``` + +#### LiteLLM as the issuer + +For deployments with no external IdP, LiteLLM signs the assertion itself with an ES256 (P-256) key. Set `anthropic_identity_source: internal_issuer` and point `anthropic_issuer_signing_key_ref` at the key rather than pasting it inline, so custody stays with your secret manager. `anthropic_issuer_ttl_seconds` defaults to 300 and cannot exceed 3600. + +```yaml +credential_list: + - credential_name: anthropic_wif + credential_values: + anthropic_identity_source: internal_issuer + anthropic_issuer_url: https://litellm.example.com + anthropic_issuer_subject: litellm-proxy + anthropic_issuer_audience: https://api.anthropic.com + anthropic_issuer_signing_key_ref: os.environ/ANTHROPIC_WIF_SIGNING_KEY + anthropic_issuer_ttl_seconds: 300 + anthropic_federation_rule_id: os.environ/ANTHROPIC_FEDERATION_RULE_ID + anthropic_organization_id: os.environ/ANTHROPIC_ORGANIZATION_ID + anthropic_service_account_id: os.environ/ANTHROPIC_SERVICE_ACCOUNT_ID + credential_info: + custom_llm_provider: anthropic + +model_list: + - model_name: claude-sonnet-4-5 + litellm_params: + model: anthropic/claude-sonnet-4-5 + litellm_credential_name: anthropic_wif +``` + +Anthropic needs the matching public key to verify what LiteLLM signs. Export it from the proxy and register it as the **inline** issuer JWKS on your federation rule: + +```shell +curl -s http://localhost:4000/credentials/anthropic_wif/jwks \ + -H "Authorization: Bearer $LITELLM_MASTER_KEY" +``` + +The endpoint is proxy-admin only and never exposes the private key, so paste the JSON it returns into the Anthropic Console rather than pointing Anthropic at the URL. A freshly registered JWKS takes about a minute before Anthropic accepts assertions signed by it. + +#### Keycloak + +For shops that already run Keycloak as the workload IdP, LiteLLM fetches the assertion with an OAuth client credentials grant. Set `anthropic_identity_source: keycloak` plus: + +```yaml + anthropic_keycloak_token_url: https://keycloak.example.com/realms/prod/protocol/openid-connect/token + anthropic_keycloak_client_id: litellm-proxy + anthropic_keycloak_auth_method: client_secret_basic # or client_secret_post + anthropic_keycloak_client_secret_ref: os.environ/KEYCLOAK_CLIENT_SECRET + anthropic_keycloak_scope: anthropic-federation +``` + +Mixing fields across variants is rejected rather than silently ignored, so a config that names `internal_issuer` while carrying Keycloak fields fails at startup instead of quietly falling back. + +### Setting it up in the UI + +The Admin UI does not offer these fields yet, so configure federation through the proxy config or the environment as shown above. A follow-up adds them to the LLM Credentials flow. + +### Token lifetime and refresh + +LiteLLM caches the minted access token per deployment and refreshes it in the background before it expires, so a request rarely waits on an exchange. Refresh is two-tier: an advisory refresh at half the token's lifetime that happens in the background while the old token keeps serving, and a mandatory one at an eighth of the lifetime that blocks. Concurrent requests for the same deployment share a single in-flight exchange rather than each minting their own, so the number of exchanges does not scale with traffic. + +Workers on the same host share the minted token through a small on-disk cache under the temp directory, readable only by the proxy's user, so a proxy running several uvicorn workers exchanges each assertion once rather than once per worker. `LITELLM_TOKEN_EXCHANGE_CACHE_DIR` moves that cache, and setting it to an empty string turns it off so every process mints on its own. Replicas on different hosts always mint their own token. + +Anthropic accepts each assertion once: a second exchange of the same JWT is denied with the opaque 401 and shows up as `jti_reused` in the rule's authentication history. That is fine for the internal issuer and Keycloak sources, which mint a fresh assertion for every exchange, but a token file or `oidc/env/` value has to rotate faster than the rule's `token_lifetime_seconds`, or the first refresh after the minted token expires fails until a new assertion shows up. Kubernetes rotates a projected service account token once 80% of its `expirationSeconds` has passed, so keep the rule's token lifetime at or above that rotation period; the defaults (a one hour projected token and a one hour rule lifetime) line up. + +### Who can configure it + +Only a proxy admin can create or change a deployment that uses workload identity federation. A team admin who otherwise manages a team-scoped deployment gets a 403, and that holds however the change is expressed: setting a federation field directly, attaching a credential that carries one by name, or changing `api_base` on a deployment that is already federated. The last one matters because `api_base` decides where the signed assertion is sent and where the minted token is presented, so it is part of the federation configuration even though it is not a federation field. + +The same rule covers the stored fallback lists on a key or team. A fallback target cannot carry a federation field, since those are merged over the deployment's own configuration when a failover happens. + +Teams keep normal access to a federated deployment. Only editing it moves to the proxy admin. + +### Restricting where assertions are sent + +The exchange only talks to `api.anthropic.com`. If you front Anthropic with a gateway, list its hostname in `LITELLM_ANTHROPIC_WIF_ALLOWED_HOSTS` (comma separated) so the signed assertion is allowed to reach it. + +```shell +export LITELLM_ANTHROPIC_WIF_ALLOWED_HOSTS="anthropic.gateway.internal" +``` + +An entry may name a port, in which case only that port is trusted and another process on the same host is not. An entry without a port trusts every port on that host. + +```shell +export LITELLM_ANTHROPIC_WIF_ALLOWED_HOSTS="anthropic.gateway.internal:8443" +``` + +The list is read from the environment only. It is never taken from a model or credential API, because `api_base` decides both where the assertion is sent and where the minted token is presented + +### Monitoring + +Token health is emitted through the standard service-logging path: `prometheus_system` sends it to Prometheus and `otel` sends it to OpenTelemetry, using the exporter settings from the [OpenTelemetry docs](../observability/opentelemetry_integration). The services are `anthropic_wif` for the exchange itself and `anthropic_wif_cache` for cache hits and misses, giving you mint counts, mint latency, and failures broken out by cause. Enable them with: + +```yaml +litellm_settings: + service_callback: ["prometheus_system", "otel"] +``` + ## Usage ```python