refactor(anthropic): exchange workload identity through the Anthropic SDK - #39948
mateo-berri wants to merge 1 commit into
Conversation
… SDK Replace the hand-rolled RFC 7523 engine in base_llm/auth/token_exchange.py with WorkloadIdentityCredentials and TokenCache from anthropic.lib.credentials. LiteLLM keeps what the SDK does not do: the identity sources, the exchange host allowlist, its own SSL settings on the exchange client, RFC 6749 error redaction with credential-echo scrubbing, and the Prometheus service metrics. The redaction and the https-only URL check move to base_llm/auth/oauth_endpoint.py, which the OpenAI client-credentials flow still shares anthropic is pinned to 1.3.0 in the proxy-runtime extra, the ci group, and the build_from_pip image, and joins the proxy-dev group so the basedpyright gate resolves the SDK types instead of measuring the new module as untyped
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Greptile SummaryThis PR replaces LiteLLM's Anthropic workload-identity exchange engine with Anthropic SDK credential helpers
Confidence Score: 4/5The implementation looks behaviorally sound, but the explicit HTTP-handler and backwards-compatibility requirements must be satisfied before merging No concrete runtime failure remains, but the dedicated client and removal of the existing public exchange API both violate repository requirements Files Needing Attention: litellm/llms/anthropic/wif_exchange.py, litellm/llms/base_llm/auth/init.py, litellm/llms/base_llm/auth/token_exchange.py
|
| Filename | Overview |
|---|---|
| litellm/llms/anthropic/wif_exchange.py | Adds the SDK-backed WIF exchange, cache, metrics bridge, redaction mapping, and a rule-violating dedicated HTTP client |
| litellm/llms/anthropic/wif.py | Routes synchronous and asynchronous WIF token resolution through the new SDK-backed exchange |
| litellm/llms/base_llm/auth/oauth_endpoint.py | Extracts shared HTTPS validation, response guards, and credential-aware OAuth error redaction |
| litellm/llms/base_llm/auth/init.py | Removes public token-exchange exports without the compatibility path required by repository rules |
| litellm/llms/base_llm/auth/token_exchange.py | Deletes the former provider-agnostic exchange engine and its import surface |
| pyproject.toml | Raises Anthropic to 1.3.0-compatible ranges and adds it to proxy development dependencies |
| tests/test_litellm/llms/anthropic/test_wif_exchange.py | Adds extensive coverage for SDK wire behavior, refreshes, caching, error redaction, metrics, and transport hardening |
Reviews (1): Last reviewed commit: "refactor(anthropic): exchange workload i..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| def new_exchange_client() -> httpx2.Client: | ||
| """The client the SDK would build for itself ignores LiteLLM's SSL settings, so the exchange | ||
| gets one built the way ``HTTPHandler`` builds its own: the same CA bundle, verification switch | ||
| and client certificate. Redirects stay off: only the bound base URL passed the host allowlist, | ||
| and a 3xx must not replay the assertion elsewhere.""" | ||
| from litellm.llms.custom_httpx.http_handler import get_ssl_configuration | ||
|
|
||
| return httpx2.Client( | ||
| verify=get_ssl_configuration(), | ||
| cert=os.getenv("SSL_CERTIFICATE", litellm.ssl_certificate), | ||
| timeout=httpx2.Timeout(EXCHANGE_TIMEOUT_SECONDS, connect=EXCHANGE_CONNECT_TIMEOUT_SECONDS), | ||
| follow_redirects=False, | ||
| ) |
There was a problem hiding this comment.
This directly constructs httpx2.Client instead of reusing LiteLLM's HTTP handlers, violating a repository requirement that must be satisfied before merging
Rule Used: What: Block any PR which writes it's own custom ht... (source)
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
| from litellm.llms.base_llm.auth.oauth_endpoint import ( | ||
| MAX_RESPONSE_BYTES, | ||
| JwtBearerTokenExchangeEngine, | ||
| default_token_exchange_engine, | ||
| redact_oauth_error_body, | ||
| validate_token_endpoint_url, | ||
| ) | ||
| from litellm.llms.base_llm.auth.types import ( | ||
| AssertionReader, | ||
| AssertionSource, | ||
| AssertionSourceError, | ||
| BodyEncoding, | ||
| ExchangeError, | ||
| ExchangeResult, | ||
| InsecureTokenUrl, | ||
| MalformedTokenResponse, | ||
| MintedToken, | ||
| SyncTokenPoster, | ||
| TokenEndpointError, | ||
| TokenExchangeSpec, | ||
| TokenTransportError, | ||
| ) |
There was a problem hiding this comment.
This removes the public token-exchange API without a compatibility path or flag, violating the repository's backwards-compatibility requirement that must be satisfied before merging
Rule Used: What: avoid backwards-incompatible changes without... (source)
LiteLLM Rust workflowWorkflow completed successfully for |
1 participant
TLDR
Problem this solves:
How it solves it:
WorkloadIdentityCredentialsandTokenCachefromanthropic.lib.credentialsrun the exchangeStacked on #39935: the base branch is
litellm_anthropic_wif_backendrather thanlitellm_internal_staging, so the diff here is only the SDK swap. Merge #39935 first, then retarget this oneUser Flow
Before: a proxy admin runs a federated Anthropic deployment and nothing fails; the token exchange is LiteLLM's own HTTP code
anthropic_identity_source, the Console ids and noapi_key, then start the proxy"model": "claude-haiku-4-5-wif"and get a normal completion backanthropic-betaheader and alitellm/<version>User-Agentlitellm_service_*samples for the exchange carrycall_typevaluescold_mint,advisory_refresh,mandatory_refreshandcache_hitAfter: the same flow succeeds the same way, and the exchange is the SDK's
anthropic-beta: oauth-2025-04-20,oidc-federation-2026-04-01andUser-Agent: anthropic-python/1.3.0call_typevaluescold_mint,refreshandcache_hitRelevant issues
Builds on #39935, which fixes #28607
Linear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Same config-only rig as #39935, run twice from this worktree on port 4000, one proxy after the other: Before at the backend branch commit, After at this branch's tip. Both boot from the same
wif_config.yaml:litellm/proxy/dev_config.yamlplus two credentials on the LiteLLM-signed issuer and two deployments on them.anthropic-wif-issuercarries placeholder Console ids (no federation rule exists in this org, so the exchange gets as far as Anthropic's token endpoint, which validates the rule id),anthropic-wif-noidscarries none. NoANTHROPIC_API_KEYorANTHROPIC_AUTH_TOKENin the environment;ISSUER_SIGNING_KEY_PEMholds a P-256 private keyEach LLM leg waits out the router cooldown from the previous refusal (65s) so the 401 is the proxy's own answer rather than a
No deployments available429. The two runs are expected to match: the point of the After run is that Anthropic's live token endpoint accepts the SDK's request (its beta header, JSON body and User-Agent) and answers with the same federation rule validation error the hand-rolled engine got, which LiteLLM maps to the same 401. A rejected header or body shape would surface as a different errorBefore (b738e13)
JWKS export for a LiteLLM-signed credential
Run
Observed
/v1/chat/completions on a federated deployment
Run
Observed
/v1/messages on a federated deployment
Run
Observed
/v1/responses on a federated deployment
Run
Observed
Federated credential missing the Console ids
Run
Observed
Federation field typed into a chat body
Run
Observed
Federation field typed into POST /model/new
Run
Observed
After (d48181f)
JWKS export for a LiteLLM-signed credential
Run
Observed
/v1/chat/completions on a federated deployment
Run
Observed
/v1/messages on a federated deployment
Run
Observed
/v1/responses on a federated deployment
Run
Observed
Federated credential missing the Console ids
Run
Observed
Federation field typed into a chat body
Run
Observed
Federation field typed into POST /model/new
Run
Observed
Type
Refactoring
Caveats (if any)
Medium
TokenCachehas no background pathcold_mintcall_typeshrinks tocold_mint,refresh,cache_hit; dashboards keyed onadvisory_refreshormandatory_refreshneed updatingLow
anthropic-betaandUser-Agentheadersasyncio.to_threadhttpx2, so the exchange client is built separately with LiteLLM's SSL settings mirrored onto itexclude-newerwindow allows; 1.4.0 clears it on 2026-09-07.venv-typecheckresolves the SDK; without it the new module measured as 39 untyped errors in the basedpyright gatetests/test_litellm/llms/__init__.pyis new so the shared test harness imports under pytest's default import modetests/test_litellm/test_lazy_imports.py::test_star_import_exports_public_apifails on the base branch too under Python 3.13, unrelated to this changeFinal Attestation