feat(auth): source request logging + spend identity from the resolved Principal - #31006
feat(auth): source request logging + spend identity from the resolved Principal#31006yassin-berriai wants to merge 3 commits into
Conversation
|
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
ef8ce8b to
54ec649
Compare
Greptile SummaryThis PR (Phase 1 of "Resolve Once, Consume Everywhere") wires the
Confidence Score: 5/5Safe to merge — all new parameters default to The change is purely additive: new optional parameters with No files require special attention.
|
| Filename | Overview |
|---|---|
| litellm/proxy/litellm_pre_call_utils.py | Adds optional principal: Optional[Principal] to four methods and threads request.state.principal to both chokepoints in add_litellm_data_to_request. Per-sub-model fallback to key is safe; all signatures are backward-compatible. |
| tests/test_litellm/proxy/test_litellm_pre_call_utils.py | Adds five new unit tests covering principal-sourced identity, key fallback without principal, per-field partial fallback, and header forwarding parity. Existing mock assertions updated to match the new principal=None default; coverage is not weakened. |
Reviews (2): Last reviewed commit: "feat(auth): source request logging + spe..." | Re-trigger Greptile
54ec649 to
140575c
Compare
|
Thanks for the review. Closed the forwarded-header gap in 140575c: add_headers_to_llm_call now takes the same Optional[Principal] and threads it through add_headers_to_llm_call_by_model_group from add_litellm_data_to_request, so the x-litellm-* identity headers forwarded to the provider follow the Principal exactly like the logged identity, with per-field key fallback when the principal is absent. Two tests cover the principal-sourced headers and the key fallback. On the embedded user_api_key_auth object: that one is intentional for Phase 1. The key object stays the carrier for budget, rate-limit, and policy state (and MCP access control), so it deliberately keeps key-sourced identity; reconciling identity reads off the embedded object belongs to Phase 2, when the authenticators normalize identity into the Principal and those consumers migrate to the flat principal-sourced fields. Tracking it there |
…itellm_principal_logging_identity # Conflicts: # litellm/proxy/litellm_pre_call_utils.py
Relevant issues
Linear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
make test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewCI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Screenshots / Proof of Fix
Verified against a live proxy on
localhost:4000backed by Postgres, hitting the real Anthropic API, running this branch withPYTHONPATHpinned to the worktree so the resolver path and this change are the code under test. Identity is resolved once at the seam intorequest.state.principal; logging and spend now source caller identity off it.Set up an organization, a team in it, a user on the team, and a team-scoped key:
Two real chat completions on that key, the second carrying an end-user:
The spend rows carry the identity resolved at the seam: the key's user, team, and org on both rows, and the end-user only on the request that sent one:
No
Principal projection at auth seam failedwarnings and no cost-tracking errors in the proxy log, and both completions returned 200. Because thePrincipalis projected from the same resolved key object today, the values match the pre-change build, so there is no regression; the behavioral guarantee that identity now follows thePrincipalrather than the key is pinned by the unit tests, where aPrincipalwhose ids differ from the key drives the logged metadataType
🆕 New Feature
Changes
This is Phase 1 of the internal Caller Identity design ("Resolve Once, Consume Everywhere"); it is the first downstream consumer wiring of the
Principalvalue type introduced in Phase 0 (PR #30887). Phase 0 resolves onePrincipalper request at the auth seam and projects it ontorequest.state.principal, but nothing downstream reads it yet, so request logging and spend attribution still derive caller identity ad hoc from theUserAPIKeyAuthkey object. That leaves identity in logs and spend potentially inconsistent with the identity resolved at the seamget_sanitized_user_information_from_keyandadd_user_api_key_auth_to_request_metadatainlitellm/proxy/litellm_pre_call_utils.pynow take an explicit, strongly-typed optionalprincipal: Optional[Principal]. When aPrincipalis present, the logged identity fields (user_api_key_user_id,user_api_key_user_email,user_api_key_team_id/user_api_key_team_alias,user_api_key_org_id/user_api_key_org_alias,user_api_key_project_id/user_api_key_project_alias,user_api_key_end_user_id) come from thePrincipal; non-identity fields (hash, alias, spend, budgets, request route, auth metadata) keep coming from the key object. Whenprincipal is None, behavior is byte-for-byte unchanged.add_litellm_data_to_requestthreadsgetattr(request.state, "principal", None)down to the chokepoint; the success spend path inherits the principal-sourced identity automatically because it reads the metadata that pre-call stuffs. Other callers that have no request principal handy passprincipal=Noneand keep current behaviorSourcing is principal-first with per-field key fallback, so a
Principalthat is present but missing a sub-model (for example nouser) never drops an id the key actually carries; it falls back to the key for that field only. Tests intests/test_litellm/proxy/test_litellm_pre_call_utils.pyassert that with aPrincipalwhose user, team, org, project, and end-user identifiers differ from the key object's, the written metadata carries the Principal's values; that withprincipal=Nonethe metadata falls back to the key object's values; and that aPrincipalpresent but missing sub-models falls back per field rather than nulling those ids