feat(anthropic): workload identity federation, pluggable identity sources, and provider-level setup - #38013
Closed
derhornspieler wants to merge 57 commits into
Closed
Conversation
derhornspieler
requested review from
ryan-crabbe-berri and
yuneng-berri
as code owners
August 23, 2026 09:12
|
|
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
![veria-ai[bot]](https://avatars.githubusercontent.com/in/1723776?s=60&v=4){kind=small}
Contributor
PR overviewAll previously flagged issues have been addressed. No open security concerns remain on this pull request. Security reviewNo open security issues remain on this pull request. Fixed/addressed: 7 · PR risk: 0/10 |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 030b0b3f19
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Contributor
Greptile SummaryThe PR adds Anthropic workload identity federation, reusable provider credentials, live model discovery, and provider-level setup across the SDK, proxy, and dashboard
Confidence Score: 5/5The PR appears safe to merge because no blocking failure remains from the previously reported issues No blocking failure remains
|
| Filename | Overview |
|---|---|
| litellm/llms/base_llm/auth/token_exchange.py | Adds the shared token-exchange engine and correctly addresses the previously reported retry and credential-reflection cases |
| litellm/llms/anthropic/wif.py | Connects Anthropic federation configuration and identity sources to the shared exchange engine |
| litellm/llms/anthropic/batches/handler.py | Propagates federation parameters and offloads synchronous credential validation for asynchronous batch retrieval |
| litellm/llms/anthropic/common_utils.py | Extends Anthropic authentication resolution and header handling for federated credentials |
| litellm/proxy/management_endpoints/model_management_endpoints.py | Adds provider discovery and administrative controls for federated model configuration |
| ui/litellm-dashboard/src/app/(dashboard)/models-and-endpoints/panels/add-provider/AddProviderPanel.tsx | Adds the provider-level Anthropic setup and discovery workflow |
Reviews (22): Last reviewed commit: "fix(proxy): keep WIF secret pointers unr..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Contributor
…oken exchange engine Adds a provider-agnostic JWT-bearer token exchange engine (litellm/llms/base_llm/auth/) with two-tier refresh, single-flight minting, negative caching, response caps, and RFC 6749 error redaction, plus the Anthropic WIF adapter and wiring: env and litellm_params config, sync and async facades, and beta-header merging fixes on the skills, files, batches, messages, and passthrough surfaces WIF is the lowest credential tier, so a deployment that does not configure it behaves exactly as before. Verified end to end against the live token endpoint from a container: both /chat/completions and /v1/messages complete over a minted sk-ant-oat01 token with no api_key configured anywhere Two protocol details were confirmed against the live endpoint rather than the docs. The exchange needs no anthropic-beta header, so none is sent. service_account_id mints fine when omitted for a single-service-account rule, so it stays optional Resolves BerriAI#28607
…nthropic providers The federation exchange derives its token endpoint from the deployment's api_base and returns an Anthropic-org credential, so it must only ever run for Anthropic itself. It did not. VertexAIAnthropicConfig inherits AnthropicModelInfo.validate_environment, and the MiniMax and Tencent /v1/messages configs inherit the Anthropic validate step, so a proxy configured with the ANTHROPIC_* federation variables POSTed the workload's OIDC assertion to those providers' own hosts, confirmed against a real deployment base Eligibility is now declared per class and read from that class's own __dict__, so a subclass written for another provider inherits nothing, and the get_auth_header facades refuse to mint unless a caller states that it authenticates against Anthropic's own API. The six federation fields join the banned request-body parameters: they select which server-side secret is read and, with api_base, where it is sent, so a caller-supplied value was an exfiltration primitive for any environment variable or mounted token file. The exchange client no longer follows redirects, since only the initial token URL is validated, and a base that already ends in /v1 no longer yields a doubled /v1/v1/oauth/token
…olled request paths An independent review of the previous commit found the narrowing incomplete. The federation fields select which server-side secret is read, and together with api_base decide where it is sent, so they are deployment decisions on every surface: - They are rejected from any request body unconditionally, ahead of the general banned-parameter check, because both client-side credential opt-ins would otherwise re-enable them. The inert workspace id keeps its existing behaviour, since Bedrock Claude Platform already accepts that spelling in a request body. - /health/test_connection takes a litellm_params object that never reached the request-body check, and its existing guard only covers os.environ references, not oidc ones. It rejects them now. - A client-redirected api_base clears the federation fields and marks the deployment, so a token is not minted for a caller-chosen host. The mark is what stops the environment-configured path, which cannot be cleared out of a dictionary Batch retrieval now threads litellm_params, so it authenticates the same ways every other Anthropic surface does rather than failing ahead of the transformation that resolves them The exchange engine always publishes a result for its single-flight entry, so an unexpected failure can no longer leave every later caller waiting on a leader that never finishes. Error bodies are rendered from structured fields only, and a body that echoes the submitted assertion is dropped rather than logged and returned. Endpoint normalization now works on the URL path, so a single-label host is left alone and a pathological base cannot exhaust the stack
…eration The workload assertion could only come from a mounted file or an environment variable, which assumes a platform that already projects one. Two more sources sit behind an explicit anthropic_identity_source discriminator, and its absence keeps today's resolver exactly as it was: - internal_issuer signs a short-lived ES256 assertion with an operator-supplied key, resolved through the usual secret reference so it can live in a secret manager. Its JWKS is exported for the operator to register with Anthropic, without which the source cannot be used at all. - keycloak fetches the assertion from a client_credentials grant, with the client secret likewise held behind a reference rather than in the configuration The engine gains an optional assertion source on the spec, so a source that needs more than a string can supply one without the reference ever carrying a secret: it stays a hash of the non-secret fields and remains what the cache keys on and what errors name. Failures carry a redacted detail, so a Keycloak hop is diagnosable rather than collapsing into one opaque message The server-owned field list is now derived from one definition, so every field added here is rejected from request bodies and cleared on a client base override without a second edit. Credential params carry the federation fields too, which is what the files, batches, and passthrough surfaces read Verified against the live token endpoint: an ES256 assertion from internal_issuer, with its exported JWKS registered on the federation issuer, mints a token. Note that a freshly registered inline JWKS takes up to about a minute to become usable
…eration Federation was configurable but only per model, and only by hand. This adds the provider-level flow the feature was missing, reusing what already exists rather than introducing parallel concepts Provider metadata gains optional credential variants: a selector plus per-variant field lists, so a provider whose credential shape branches describes that in the same metadata every provider already publishes, and the existing generic renderer drives it with no provider-specific branch. Anthropic publishes five, one per way of authenticating. Fields holding a secret reference render as text, not as password inputs that would invite pasting the secret itself Model discovery could not see a credential configured through litellm_params, only through the environment, so a federated provider had nothing to discover. Providers now expose discovery that receives the deployment params, and an admin-only endpoint discovers a provider's models from a stored credential by name. The credential fields are never accepted in that request, since choosing which server-side secret is read is not a caller's decision Enable and disable reuse the existing blocked flag rather than a second notion of the same thing, which meant letting model creation persist it so a model can be created already disabled instead of being created live and then paused. Alternate names reuse the existing public name and group alias. One credential now feeds many models, so switching a credential's auth variant has to be able to remove the previous variant's fields, which credential updates could not express before The Admin UI gains an Add Provider flow over all of it: pick the provider and auth method, fill that method's fields, export the JWKS when the identity is self-signed, discover the provider's models, then enable, rename, and alias them before creating. Two pre-existing UI defects surfaced while building it: a variant-capable provider could render through the stale flat field list before its metadata loaded and leave a validation rule behind that blocked submission, and the credential editor left the previous variant's fields in place when the variant changed
…ject /model/block and /model/unblock declare the proxy model table as their response model, and FastAPI validates the returned row against it with from_attributes. The before-validator that parses the JSON string columns assumed a mapping, so it called .get on the row object and both endpoints answered 500 even though the block itself had already been applied The validator now passes an object through untouched and keeps parsing the string columns it exists for, which is the case the database actually produces. Found while exercising the provider setup's enable and disable toggle against a live proxy
…types The discovery and JWKS routes were missing from the generated types. Regenerating on a developer machine can drop the enterprise routes when that package is not importable, which turns the sync check into a large deletion; this spec was dumped from the built container image, so the diff is only the two added endpoints
…-encode Keycloak credentials The retry path re-read the assertion from its source to decide whether an error body reflected it back. A rotating source (a token file rewritten between attempts, a Keycloak fetch that mints a fresh token) hands back a different value, so the assertion the failing request carried stayed in the error text. _Unauthorized now carries the assertion the attempt used and the redaction runs against that, which also drops one extra read of the secret per failure Separately, RFC 6749 2.3.1 wants the client id and secret form-urlencoded before the base64, so a reserved character in either one no longer corrupts what the far side decodes back out of Basic auth
…g the event loop The async file and batch handlers called the provider's sync validate_environment directly. For the workload identity tier that hook performs a blocking token exchange, so a mint stalled the whole loop. They now go through one facade that awaits the provider's async hook when it has one and offloads the sync hook to a worker thread otherwise, so every other surface keeps its existing behaviour Batch retrieval also copies the litellm_params it hands down rather than sharing the caller's dict, and get_anthropic_headers moves its credential selection into a helper so the four auth tiers read as four branches instead of one nested chain
… fields These fields pick which server-side secret is read and where the resulting assertion is sent, so a team admin who can otherwise manage a team-scoped deployment must not be able to set them. The create, update and patch model paths and both credential write paths now reject them for anyone below proxy admin, mirroring the existing blocked-flag gate. The field list is derived from anthropic_wif_litellm_params rather than copied, so a new federation field is covered the day it is added Also fixes two things CI caught: credential_values_to_delete is a PATCH instruction rather than part of the credential, so it stays out of dumps that feed config loading and the Prisma write, and the provider discovery route is registered in the backend allowlist
…es for failed rows The tab list put add-provider ahead of auto-routers, which reordered the tabs an existing test pins. It now sits after auto-routers, where the admin-only tabs start The wizard also wrote a model_group_alias entry for every discovered row it tried, including rows whose model creation had just failed, leaving an alias pointing at a deployment that does not exist. Alias additions are now taken from the rows that actually got created
Both posters built their HTTPHandler inline, so nothing exercised the guarantees that matter there: redirects stay disabled, an HTTPStatusError comes back as its response rather than escaping, and a None response raises instead of being dereferenced. Each now takes a handler factory that defaults to the real one, which is the same injection seam the rest of this package already uses for its poster and secret reader Coverage for litellm/llms/base_llm/auth goes from 90% to 97%: client_credentials from 79% to 98%, with only its assert_never arm left, and token_exchange from 90% to 96%
…l schema Excluding credential_values_to_delete from dumps makes pydantic emit separate input and output schemas for CredentialItem, since the field is accepted on a write and absent from every read. The generated types now say so: the credential read routes return the output shape, which no longer carries the PATCH-only field, and only the write route accepts it
derhornspieler
force-pushed
the
litellm_anthropic_wif_oidc_28607
branch
from
August 23, 2026 17:34
d965781 to
ad03777
Compare
Author
|
@greptileai Pushed fixes for the retry redaction and the blocking files and batches paths. Please re-review at ad03777 |
Author
|
@codex review Both P1 findings are fixed: the retried assertion is now redacted, and Keycloak client credentials are form-encoded |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ad03777789
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
… litellm_anthropic_wif_oidc_28607 # Conflicts: # basedpyright-code-budget.json # litellm/proxy/credential_endpoints/endpoints.py # litellm/proxy/management_endpoints/model_management_endpoints.py # tests/test_litellm/router_utils/test_fallback_event_handlers.py # type-discipline-budget.json
Author
|
@greptile-apps review |
The merge with the moved base pushed three budget gates over their ceilings. credential_hydration collected the per-credential field lookups with a list comprehension over an async call, which LIT002 reads as building a mutable value by accumulation. Replaced with asyncio.gather plus chain.from_iterable, so the lookups now also run concurrently instead of one after another. The credential form helper test carried an inline object literal that put no-large-inline-object-arg one over its 560 ceiling. Hoisted it to a named const. Two patch sites in the pass-through relay test were counted by TQ008. Both are genuine boundaries, the http client the test asserts bytes against and the logging hook that would otherwise want a database, so they carry suppressions naming the reason rather than being restructured.
The last merge rebuilt litellm/proxy/_experimental/out, so 480 generated files landed in the diff and pushed it to 603 files, past the 500 file ceiling the review bot will look at. That bundle is refreshed by its own periodic chore commit rather than by feature branches, so this restores the directory to the base branch state. The dashboard source changes stay; the compiled output comes back with the next scheduled artifact refresh.
Author
|
@greptile-apps review Dropped the regenerated dashboard bundle that a merge pulled in, so this is back to 123 files and under your limit |
The paginated /v1/models fetch handed the shared http client a read-only mapping for its query params. That client merges the URL's own query string in by mutating the mapping it is given, and a mappingproxy has no update, so the call raised AttributeError as soon as after_id was set. Any org with more models than fit on one page could not discover them at all. after_id now rides the URL instead, which is the same mechanism the client uses to carry query params, so nothing needs to mutate. The existing pagination test missed this because its stub client only recorded params and never merged them the way the real one does. The new test drives the real HTTPHandler over a mock transport, so the second page goes through the code that actually mutates. It fails with the original AttributeError against the previous implementation. Also clears the basedpyright reportArgumentType errors this branch added: make_spec now takes typed keyword arguments rather than splatting an untyped dict, both token_response helpers build their body in one shot instead of assigning an int into a str-inferred dict, the duplicated credential decrypt block is one helper, and HTTPHandler.get accepts a Mapping for headers, which it only forwards.
…lats Two test call sites splatted a plain dict of strings into a strongly typed callable. Because every parameter then looks like it might receive a str, basedpyright raised one error per parameter: 118 for the GenericLiteLLMParams construction and 30 for the completion call, 148 in a file that otherwise has 14. Passing the values as keywords types them properly and drops the file to those 14 pre-existing ones. The leak assertion that walked the old kwargs dict now walks a tuple of the same field names, so it still proves none of them reach the request body. HTTPHandler.get takes Mapping[str, Any] for headers rather than the Mapping[str, str] of the previous commit. The parameter was a bare dict, which accepts any value type, so narrowing it to str rejected existing callers that pass looser mappings and added six errors of its own.
_create_deployment merges its untyped params dict with the zeroed PTU pricing mapping and spreads the result into LiteLLM_Params. The merge widens every value to Unknown | float | tuple[()] | Mapping[str, float], and because a spread is checked against each field in turn, basedpyright raised one error per field: 183 on that one line. This branch adds eighteen anthropic federation fields to that model, so it inherited eighteen more errors on a line it does not touch, which is the whole of the reportArgumentType budget breach. The same effect landed two more in the azure passthrough transformation. Naming the merged value as the heterogeneous config mapping it is fixes the line rather than the symptom. LiteLLM_Params still validates every field at runtime, so nothing is loosened that was previously enforced. The file drops from 402 of these errors to 237, which puts the rule below its base count instead of over its ceiling.
Author
|
@greptile-apps review Three commits since your last pass: a model discovery pagination fix, typed keyword args in the tests, and a type annotation on the params spread in router.py |
…itellm_anthropic_wif_oidc_28607 # Conflicts: # litellm/llms/anthropic/common_utils.py # litellm/llms/anthropic/experimental_pass_through/messages/transformation.py # litellm/llms/anthropic/skills/transformation.py # litellm/proxy/management_endpoints/model_management_endpoints.py # litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py # tests/test_litellm/llms/anthropic/experimental_pass_through/adapters/test_anthropic_experimental_pass_through_adapters_transformation.py # tests/test_litellm/llms/anthropic/experimental_pass_through/test_reasoning_effort_fields.py # tests/test_litellm/llms/anthropic/test_anthropic_common_utils.py # tests/test_litellm/proxy/management_endpoints/test_model_management_endpoints.py # tests/test_litellm/proxy/pass_through_endpoints/test_llm_pass_through_endpoints.py
Contributor
… credential The reflection scan only ever compared eight-character windows of the response against the secret, so a secret with fewer credential characters than that could never match once an endpoint echoed it percent-encoded or split up, and the verbatim check needs the raw form. Cap the run at the secret's length so a short client secret is compared whole in every wire shape.
…encent messages Upstream e14f485 makes validate_anthropic_messages_environment raise AuthenticationError when no credential resolves instead of sending the request unauthenticated. The two regression tests that prove MiniMax and Tencent never mint an Anthropic federation token now assert that raise, which a wrongly enabled federation path would still break by attaching a bearer instead.
…age audit
GET /credentials/{credential_name}/jwks serves the public key set of an internal
issuer credential for the Add Provider wizard, a derived read-only view rather
than Terraform-managed state.
Author
|
@greptile-apps review Fixed since the staging merge: the short-secret redaction gap, the MiniMax and Tencent tests for upstream's missing-key raise, and the JWKS coverage allowlist. |
…async create_file and create_batch
Contributor
Contributor
|
bugbot run |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit f2240df. Configure here.
| auth_header: Final = self.anthropic_model_info.get_auth_header(api_key, api_base) | ||
| auth_header: Final = await self.anthropic_model_info.aget_auth_header( | ||
| api_key, api_base, allow_workload_identity=True | ||
| ) |
Contributor
There was a problem hiding this comment.
Files handler drops deployment WIF params
Medium Severity
afile_content now mints a federation token but calls aget_auth_header without litellm_params. Deployment-scoped federation fields never reach the resolver, so batch-result fetches that go through this handler still fail unless the same values happen to be in the process environment.
Reviewed by Cursor Bugbot for commit f2240df. Configure here.
Contributor
There was a problem hiding this comment.
This handler has no runtime caller; anthropic file_content routes through get_provider_files_config and base_llm_http_handler.retrieve_file_content, which receives litellm_params.
21 tasks
Contributor
10 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TLDR
Problem this solves:
How it solves it:
User Flow
Before: a platform team with a no-static-secrets policy cannot point LiteLLM at an Anthropic federation rule
model: anthropic/claude-haiku-4-5{"model": "claude-haiku-4-5", "messages": [...]}Missing Anthropic API Key - A call is being made to anthropic but no key is set either in the environment variables or via paramsAfter: the same team configures the provider once and every discovered model authenticates without a static key
fdrl_..., an org id, and ansvac_...os.environ/KC_CLIENT_SECRET, never a pasted secretRelevant issues
Fixes #28607
Docs: BerriAI/litellm-docs#1000
Linear ticket
Resolves LIT-6107
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Federated end to end (contributor's run at 197fd1e, before at fde3075)
Four models on
anthropic/, noapi_keyanywhere, federation ids from the environment, and the identity token projected at a path inside the OIDC file allowlist:curl -s localhost:4100/v1/chat/completions -d '{"model":"claude-haiku-4-5-wif","messages":[{"role":"user","content":"Reply with exactly: WIF OK"}],"max_tokens":16}'{"error":{"message":"litellm.AuthenticationError: Missing Anthropic API Key - A call is being made to anthropic but no key is set either in the environment variables or via params. ..."}}. After:content: WIF OK | model: claude-haiku-4-5-wif | tokens: 20/v1/messageswithclaude-sonnet-5-wif. Before:{"type":"error","error":{"type":"authentication_error","message":"x-api-key header is required"}}. After:content: WIF OK | usage: 19 in / 7 outcurl -s localhost:4100/v1/messages/count_tokens -d '{"model":"claude-haiku-4-5-wif","messages":[{"role":"user","content":"count these tokens please"}]}'. Before: no credential, the count never reaches Anthropic. After:{"input_tokens":11}, Anthropic's own countcurl -s localhost:4100/v1/chat/completions -d '{"model":"claude-haiku-4-5-wif", ..., "anthropic_identity_token":"attacker-supplied"}'. Before: the field is neither refused nor honoured and the call fails for want of a key. After:{"error":{"message":"Authentication Error, Rejected Request: anthropic_identity_token is a server-owned workload identity federation parameter and cannot be set in a request body; configure it on the deployment instead.","type":"auth_error","code":"401"}}curl -s localhost:4100/public/providers/fields. Before:credential_variants: None. After:credential_variants: ['api_key', 'wif_token', 'wif_token_file', 'wif_internal_issuer', 'wif_keycloak']The federated 200 legs were not re-run at the tip: the federation rule, service account, and identity token live only in a Claude Console organization the reviewer cannot provision, so the review runs below cover every leg that does not need a live federation rule. The tip changes nothing on the path those legs exercise beyond the three fixes proven below
Review run at f2240df vs base e585aab, carried to 57b9e94 since that commit only touches the dashboard wizard (two proxies,
--num_workers 2each, real Anthropic key on the control models, real spend)Same config, same 30 requests in the same order on both sides, S01 to S23 in the run log. Identical on both sides: liveliness and readiness,
/v1/chat/completions,/v1/messages,/v1/responses,/v1/messages/count_tokensand/anthropic/v1/messagespassthrough on a static-key deployment and on an env-key deployment (all 200,pong), key generate, chat, key info spend and delete, the spend log row,/v1/model/info,/v1/models,/ui/,/health. The differences are the feature:POST /v1/chat/completionswith"anthropic_federation_rule_id":"fdrl_x"in the body (S10). Base:400 AnthropicException - invalid_request_errorafter the field reached Anthropic as an unknown parameter. Tip:401 Authentication Error, Rejected Request: anthropic_federation_rule_id is a server-owned workload identity federation parameter and cannot be set in a request body; configure it on the deployment instead.POST /provider/models/discoverwith{"custom_llm_provider":"anthropic"}(S13). Base:404. Tip:200with the live model list; the same call carryinganthropic_federation_rule_id(S13b) is refused with the same401POST /model/newcarrying federation fields inline, as a team admin (S15e) and as the proxy admin (S15h): both401naming the field. A team admin onPOST /credentials(S15g):401 Only proxy admin can be used to generate, delete, update info for new keys/users/teams. Route=/credentials. A team admin creating a plain deployment (S15f) still gets200GET /public/providers/fields(S12). Base: nocredential_variantskey. Tip:credential_variants: [api_key, wif_token, wif_token_file, wif_internal_issuer, wif_keycloak]on the anthropic entry, every other provider unchangedANTHROPIC_API_KEYis set in the proxy environment (S11) answers200 pongon both sides: the static key wins over federation, see CaveatsDefects found in review, before at a7ed600 and after at f2240df, the dashboard defect after at 57b9e94 (proxy booted without
ANTHROPIC_API_KEY,--num_workers 2,router_settings: disable_cooldowns: true)Config:
claude-wif-keycloak(keycloak,anthropic_keycloak_client_secret_ref: os.environ/KC_SECRET, token url pointing at an unreachablehttps://127.0.0.1:9/...),claude-wif-tokenfile(token file insideLITELLM_OIDC_ALLOWED_CREDENTIAL_DIRS),claude-wif-issuer(internal_issuer inline,anthropic_issuer_signing_key_ref: os.environ/ISSUER_KEYholding a P-256 PEM),claude-wif-issuer-cred(litellm_credential_namepointing at acredential_listentry mirroring the docs example), federation ids asos.environ/references, every one a placeholder (fdrl_rigbogus...) since no live rule is availablePOST /v1/messagesonclaude-wif-keycloak(D03). Before:401 anthropic_identity_source is 'keycloak', but anthropic_issuer_audience, anthropic_issuer_signing_key_ref, anthropic_issuer_subject, anthropic_issuer_ttl_seconds, anthropic_issuer_url belongs to a different identity source, so the endpoint could never federate:/v1/messageshands the resolver every deployment field with the unset ones asNone, and the resolver rejected them as foreign. After: the request reaches the identity source,401 Could not obtain the OIDC identity token (unreadable) from oidc/keycloak/...: could not reach the keycloak token endpoint https://127.0.0.1:9/realms/rig/protocol/openid-connect/token: ConnectError. The same flip onclaude-wif-issuer-cred(D09) and on a credential-backed model (D13)POST /v1/chat/completionsonclaude-wif-keycloak(D02) and onclaude-wif-issuer(D07) andclaude-wif-issuer-cred(D08). Before:401 ... keycloak client secret <withheld: not a secret reference> could not be readand401 ... internal_issuer signing key <withheld: not a secret reference> could not be read: config.yaml loading expandedos.environ/KC_SECRETinto the raw secret, and the identity source, which dereferences that pointer itself at use time, then saw a value that is not a reference. After: D02 reaches Keycloak (ConnectErrorfrom the unreachable rig endpoint), D07 and D08 mint and send the assertion and get Anthropic's own answer,401 The token endpoint returned HTTP 400: error: invalid_request_error - federation_rule_id is not a well-formed fdrl_ tagged ID, which is exactly the placeholder rule id. The token-file model (D04, D05) returns that same Anthropic answer on both sides, and a credential created throughPOST /credentials(D10 to D13) was never affected because the credentials API never expanded the referencecreate_fileandcreate_batchvalidated the provider credential on the event loop before dispatching, so a federated token exchange during an async file upload or batch create blocked every other request on that worker. No curl shows this deterministically; the regression tests intests/test_litellm/llms/custom_httpx/test_llm_http_handler.pyassert the async hook is awaited (and a sync hook runs off the loop thread) and fail on the previous handlerAddProviderPanel.integration.test.tsxtypes a key, switches provider, and asserts the new credential is created with only the key typed for it, failing on the previous panel with the first keyObserved alongside, none caused by this PR:
/v1/messages/count_tokensmaps a federation auth failure to 500, pre-existing/model/newmodel can miss its/credentialscredential on the other worker, pre-existingANTHROPIC_API_KEYsilently wins over deployment federation, by designblockedflag on keys, pre-existingType
New Feature
Caveats (if any)
Low
api_keyor an ambientANTHROPIC_API_KEY/ANTHROPIC_AUTH_TOKENtakes precedence over the deployment's federation fields, silently (S11): the same precedence every provider uses, and the docs PR gets a note; unset the env var on a no-static-secrets proxyLITELLM_OIDC_ALLOWED_CREDENTIAL_DIRS, and the token exchange only talks to api.anthropic.com unless the operator adds hosts toLITELLM_ANTHROPIC_WIF_ALLOWED_HOSTS, so a gateway or test double needs that setPOST /model/newfor the proxy admin too, not only for team admins: the sanctioned paths are a named credential (POST /credentialsthenlitellm_credential_name) or config.yaml. Editing a federated deployment stays proxy-admin only, including changing its api_base or attaching a credential that carries federation valuesCredentialItem-InputandCredentialItem-Outputinstead of oneCredentialItem, becausecredential_values_to_deleteis excluded from serialization. The wire shape is unchanged; a generated client pinned to the old schema name regenerates. Keeping one name would mean splitting the request and response models, more churn than the rename costsapi_basegets a different synthesized per-request model id than before, since the federation disable flag now rides those params. Cosmetic, the id is not persistedanthropic_workspace_idis refused in a request body even on a Bedrock deployment, where it is harmless; it stays refused for uniformitytags, so it cannot change a federation field/public/agents/fieldsdoes not list the federation variants, so the agents page offers API key only; the models page doesFinal Attestation