fix: populate identity fields in proxy admin JWT early-return path#21169
fix: populate identity fields in proxy admin JWT early-return path#21169ishaan-jaff merged 2 commits intomainfrom
Conversation
When is_proxy_admin is True, the UserAPIKeyAuth early-return now includes user_id, team_id, team_alias, team_metadata, org_id, and end_user_id resolved from the JWT. Previously only user_role and parent_otel_span were set, causing blank Team Name and Internal User in Request Logs UI.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile OverviewGreptile SummaryThis PR fixes a bug where JWT-authenticated proxy admin users had blank Team Name and Internal User fields in the Request Logs UI. The root cause was that the
Confidence Score: 4/5
|
| Filename | Overview |
|---|---|
| litellm/proxy/auth/user_api_key_auth.py | Populates identity fields (user_id, team_id, team_alias, team_metadata, org_id, end_user_id) in the proxy admin JWT early-return path. The fix correctly mirrors the non-admin path for identity fields but intentionally omits rate-limiting fields (team_tpm_limit, team_rpm_limit, team_models) since admin users skip budget checks. |
| tests/test_litellm/proxy/auth/test_user_api_key_auth.py | Adds two unit tests for proxy admin JWT auth identity fields. Tests validate UserAPIKeyAuth constructor behavior but don't test the actual auth flow (they don't invoke _user_api_key_auth_builder), reducing confidence that the production code path is exercised. |
Flowchart
flowchart TD
A[JWT Token Received] --> B[JWTAuthManager.auth_builder]
B --> C{is_proxy_admin?}
C -->|Yes| D[Return UserAPIKeyAuth<br/>with identity fields<br/>- user_id, team_id, team_alias<br/>- team_metadata, org_id, end_user_id<br/>- user_role=PROXY_ADMIN]
C -->|No| E[Build valid_token with<br/>full team limits & models]
E --> F[Run common_checks<br/>budget, model access, etc.]
F --> G[Return UserAPIKeyAuth]
D --> H[Skip common_checks<br/>and rate limit fields]
style D fill:#90EE90
style H fill:#FFCC80
Last reviewed commit: 96db536
| def test_proxy_admin_jwt_auth_includes_identity_fields(): | ||
| """ | ||
| Test that the proxy admin early-return path in JWT auth populates | ||
| user_id, team_id, team_alias, team_metadata, org_id, and end_user_id. | ||
|
|
||
| Regression test: previously the is_proxy_admin branch only set user_role | ||
| and parent_otel_span, discarding all identity fields resolved from the JWT. | ||
| This caused blank Team Name and Internal User in Request Logs UI. | ||
| """ | ||
| from litellm.proxy._types import LiteLLM_TeamTable, LitellmUserRoles, UserAPIKeyAuth | ||
|
|
||
| team_object = LiteLLM_TeamTable( | ||
| team_id="team-123", | ||
| team_alias="my-team", | ||
| metadata={"tags": ["prod"], "env": "production"}, | ||
| ) | ||
|
|
||
| # Simulate the proxy admin early-return path (user_api_key_auth.py ~line 586) | ||
| result = UserAPIKeyAuth( | ||
| api_key=None, | ||
| user_role=LitellmUserRoles.PROXY_ADMIN, | ||
| user_id="user-abc", | ||
| team_id="team-123", | ||
| team_alias=( | ||
| team_object.team_alias if team_object is not None else None | ||
| ), | ||
| team_metadata=team_object.metadata if team_object is not None else None, | ||
| org_id="org-456", | ||
| end_user_id="end-user-789", | ||
| parent_otel_span=None, | ||
| ) | ||
|
|
||
| assert result.user_role == LitellmUserRoles.PROXY_ADMIN | ||
| assert result.user_id == "user-abc" | ||
| assert result.team_id == "team-123" | ||
| assert result.team_alias == "my-team" | ||
| assert result.team_metadata == {"tags": ["prod"], "env": "production"} | ||
| assert result.org_id == "org-456" | ||
| assert result.end_user_id == "end-user-789" | ||
| assert result.api_key is None |
There was a problem hiding this comment.
Tests don't exercise the actual auth code path
Both new tests construct a UserAPIKeyAuth object directly rather than invoking _user_api_key_auth_builder with a mocked JWT flow. This means they validate that UserAPIKeyAuth accepts the fields (which is trivially true since these are existing Pydantic fields), but don't verify that the production code in user_api_key_auth.py:586-603 actually passes those values correctly.
Consider adding a test that mocks the JWT auth flow (similar to test_proxy_admin_expired_key_from_cache at line 243) and asserts that the returned UserAPIKeyAuth has identity fields populated when is_proxy_admin=True.
| team_metadata=team_object.metadata | ||
| if team_object is not None | ||
| else None, |
There was a problem hiding this comment.
Inconsistent parenthesization with team_alias
The team_metadata ternary expression lacks the explicit parentheses that team_alias (line 592-596) uses. While this works correctly due to operator precedence, the inconsistent style reduces readability within the same block.
| team_metadata=team_object.metadata | |
| if team_object is not None | |
| else None, | |
| team_metadata=( | |
| team_object.metadata | |
| if team_object is not None | |
| else None | |
| ), |
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
…ctions (#21192) * Access groups UI * new badge changes * adding tests * fix: add custom_body parameter to endpoint_func in create_pass_through_route (#20849) * fix: add custom_body parameter to endpoint_func in create_pass_through_route The bedrock_proxy_route calls `endpoint_func(custom_body=data)` to pass a pre-parsed, SigV4-signed request body. However, the `endpoint_func` closure created by `create_pass_through_route` does not accept a `custom_body` keyword argument, causing: TypeError: endpoint_func() got an unexpected keyword argument 'custom_body' Add `custom_body: Optional[dict] = None` to both `endpoint_func` definitions (adapter-based and URL-based). In the URL-based path, when `custom_body` is provided by the caller, use it instead of re-parsing the body from the raw request. Fixes #16999 * Add tests for custom_body handling in create_pass_through_route Address reviewer feedback on PR #20849: - Document why the adapter-based endpoint_func accepts custom_body for signature compatibility but does not forward it (the underlying chat_completion_pass_through_endpoint does not support it). - Add test_create_pass_through_route_custom_body_url_target: verifies that when a caller (e.g. bedrock_proxy_route) supplies custom_body, it takes precedence over the body parsed from the raw request. - Add test_create_pass_through_route_no_custom_body_falls_back: verifies that the default path (no custom_body) correctly uses the request-parsed body, preserving existing behavior. Both tests are fully mocked following the project's CONTRIBUTING.md guidelines and the patterns established in the existing test file. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: themavik <themavik@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com> * change to model name for backwards compat * addressing comments * allow editing of access group names * fix: populate identity fields in proxy admin JWT early-return path (#21169) * fix: populate identity fields in proxy admin JWT early-return path When is_proxy_admin is True, the UserAPIKeyAuth early-return now includes user_id, team_id, team_alias, team_metadata, org_id, and end_user_id resolved from the JWT. Previously only user_role and parent_otel_span were set, causing blank Team Name and Internal User in Request Logs UI. * test: add unit tests for proxy admin JWT identity fields * bump: version 0.4.36 → 0.4.37 * migration + build files * Add pyroscope for observability (#21167) * Pyroscope: require PYROSCOPE_APP_NAME and PYROSCOPE_SERVER_ADDRESS, add UTF-8 locale hint - No defaults for PYROSCOPE_APP_NAME or PYROSCOPE_SERVER_ADDRESS; fail at startup if unset when Pyroscope is enabled - Set LANG/LC_ALL to C.UTF-8 when unset to reduce malformed_profile (invalid UTF-8) rejections - Startup message suggests PYTHONUTF8=1 if server rejects profiles - Simplify LITELLM_ENABLE_PYROSCOPE in config_settings; document Pyroscope env vars as required with no default - Add pyroscope_profiling to sidebar (Alerting & Monitoring) - pyproject.toml: pyroscope-io as required dep on non-Windows (marker), in proxy extra * proxy: add PYROSCOPE_SAMPLE_RATE env, use verbose logging, fix int type - Add optional PYROSCOPE_SAMPLE_RATE env (integer, no default) - Pass sample_rate to pyroscope.configure() as int for pyroscope-io - Replace print with verbose_proxy_logger (info/warning) - Document PYROSCOPE_SAMPLE_RATE in config_settings.md * Address Greptile PR feedback: Pyroscope optional, docs, tests, docstring - pyproject.toml: mark pyroscope-io as optional=true (proxy extra only) - Add docs/my-website/docs/proxy/pyroscope_profiling.md (fix broken sidebar link) - Add tests/test_litellm/proxy/test_pyroscope.py for _init_pyroscope() - proxy_server: fix _init_pyroscope docstring (required server/app name, sample rate as int) * Update litellm/proxy/proxy_server.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * fix(model_info): Add missing tpm/rpm for Gemini models (#21175) Several Gemini models (TTS, native-audio, robotics, gemma) were missing tpm/rpm values, causing test_get_model_info_gemini to fail. Added conservative default values (tpm=250000, rpm=10) for preview models. gemini-2.5-flash-preview-tts gets tpm=4000000, rpm=10. Co-authored-by: OpenClaw <openclaw@users.noreply.github.com> * fix(ci): Fix ruff lint error - unused import in vertex_ai_ingestion (#21178) Co-authored-by: shin-bot-litellm <shin-bot-litellm@users.noreply.github.com> * fix(ci): Fix mypy type errors across 6 files (#21179) - vertex_ai/gemini: fix TypedDict assignment via explicit dict cast - mcp_server: convert MutableMapping scope to dict for type safety - pass_through_endpoints: simplify custom_body logic to fix type narrowing - vector_store_endpoints: add Any annotation for dynamic hook return - responses transformation: use dict() for Reasoning and setattr for dynamic field - zscaler_ai_guard: add assert for api_base None check Co-authored-by: shin-bot-litellm <shin-bot-litellm@users.noreply.github.com> * fix(ci): Fix E2E login button selector - use exact match (#21176) * fix(ci): Fix ruff lint error - unused import Remove unused 'cast' import in vertex_ai_ingestion.py (ruff F401) * fix(ci): Fix E2E login button selector - use exact match Login button selector now matches both 'Login' and 'Login with SSO', causing strict mode violation. Use { exact: true } to match only 'Login'. --------- Co-authored-by: OpenClaw <openclaw@users.noreply.github.com> * fix(mypy): Fix type errors across multiple files (#21180) - vertex_ai/gemini/transformation.py: Fix TypedDict assignment via dict alias - mcp_server/server.py: Convert ASGI scope to dict for type compatibility - pass_through_endpoints.py: Add explicit Optional[dict] type annotation - vector_store_endpoints/endpoints.py: Add Any type for dynamic proxy hook - responses transformation.py: Use dict(Reasoning()) and setattr for compatibility - zscaler_ai_guard.py: Add assert for api_base nullability Co-authored-by: OpenClaw <openclaw@users.noreply.github.com> * [Guardrails] Add guardrail pipeline support for conditional sequential execution (#21177) * Add pipeline type definitions for guardrail pipelines PipelineStep, GuardrailPipeline, PipelineStepResult, PipelineExecutionResult with validation for actions (allow/block/next/modify_response) and modes. * Export pipeline types from policy_engine types package * Add optional pipeline field to Policy model * Add pipeline executor for sequential guardrail execution * Parse pipeline config in policy registry * Add pipeline validation in policy validator * Add pipeline resolution and managed guardrail tracking * Resolve pipelines and exclude managed guardrails in pre-call * Integrate pipeline execution into proxy pre_call_hook * Add test guardrails for pipeline E2E testing * Add example pipeline config YAML * Add unit tests for pipeline type definitions * Add unit tests for pipeline executor * Update litellm/proxy/policy_engine/pipeline_executor.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * Update litellm/proxy/utils.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * Add pipeline flow builder UI for guardrail policies (#21188) * Add pipeline type definitions for guardrail pipelines PipelineStep, GuardrailPipeline, PipelineStepResult, PipelineExecutionResult with validation for actions (allow/block/next/modify_response) and modes. * Export pipeline types from policy_engine types package * Add optional pipeline field to Policy model * Add pipeline executor for sequential guardrail execution * Parse pipeline config in policy registry * Add pipeline validation in policy validator * Add pipeline resolution and managed guardrail tracking * Resolve pipelines and exclude managed guardrails in pre-call * Integrate pipeline execution into proxy pre_call_hook * Add test guardrails for pipeline E2E testing * Add example pipeline config YAML * Add unit tests for pipeline type definitions * Add unit tests for pipeline executor * Add pipeline column to LiteLLM_PolicyTable schema * Add pipeline field to policy CRUD request/response types * Add pipeline support to policy DB CRUD operations * Add PipelineStep and GuardrailPipeline TypeScript types * Add Zapier-style pipeline flow builder UI component * Integrate pipeline flow builder with mode toggle in policy form * Add pipeline display section to policy info view * Add unit tests for pipeline in policy CRUD types * Refactor policy form to show mode picker first with icon cards * Add full-screen FlowBuilderPage component for pipeline editing * Wire up full-screen flow builder in PoliciesPanel with edit routing * Restyle flow builder to match dev-tool UI aesthetic * Restyle flow builder cards to match reference design * Update step card to expanded layout with stacked ON PASS / ON FAIL sections * Add end card to flow builder showing return to normal control flow * Add PipelineTestRequest type for test-pipeline endpoint * Export PipelineTestRequest from policy_engine types * Add POST /policies/test-pipeline endpoint * Add testPipelineCall networking function * Add PipelineStepResult and PipelineTestResult types * Add test pipeline panel to flow builder with run button and results display * Fix pipeline executor: inject guardrail name into metadata so should_run_guardrail allows execution * Update litellm/proxy/policy_engine/pipeline_executor.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * Update litellm/proxy/utils.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * Update litellm/proxy/policy_engine/policy_endpoints.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * Update litellm/proxy/policy_engine/pipeline_executor.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * fix(responses-bridge): extract list-format system content into instructions When system message content is a list of content blocks (e.g. [{"type": "text", "text": "..."}]) instead of a plain string, the responses API bridge was passing it through as a role: system message in the input items. APIs like ChatGPT Codex reject this with "System messages are not allowed". This happens when requests come through the Anthropic /v1/messages adapter, which converts system prompts into list-format content blocks in the OpenAI chat completions format. Fix: extract text from list content blocks and concatenate into the instructions parameter, matching the existing behavior for string system content. * test: add tests for system message extraction in responses bridge Add three tests for convert_chat_completion_messages_to_responses_api: - String system content → instructions - List-format content blocks → instructions (the bug this PR fixes) - Multiple system messages (mixed string and list) concatenated * fix: add warning log for unexpected system content types Address review feedback: add an else clause that logs a warning for any system content that is neither str nor list, rather than silently dropping it. --------- Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com> Co-authored-by: The Mavik <179817126+themavik@users.noreply.github.com> Co-authored-by: themavik <themavik@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: Alexsander Hamir <alexsanderhamirgomesbaptista@gmail.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> Co-authored-by: shin-bot-litellm <shin-bot-litellm@berri.ai> Co-authored-by: OpenClaw <openclaw@users.noreply.github.com> Co-authored-by: shin-bot-litellm <shin-bot-litellm@users.noreply.github.com>
…erriAI#21169) * fix: populate identity fields in proxy admin JWT early-return path When is_proxy_admin is True, the UserAPIKeyAuth early-return now includes user_id, team_id, team_alias, team_metadata, org_id, and end_user_id resolved from the JWT. Previously only user_role and parent_otel_span were set, causing blank Team Name and Internal User in Request Logs UI. * test: add unit tests for proxy admin JWT identity fields
Summary
is_proxy_adminis True during JWT auth, theUserAPIKeyAuthearly-return now includesuser_id,team_id,team_alias,team_metadata,org_id, andend_user_idresolved from the JWTuser_roleandparent_otel_spanwere set, causing blank Team Name and Internal User in the Request Logs UI for JWT-authenticated proxy adminsTest plan
test_proxy_admin_jwt_auth_includes_identity_fields— verifies all identity fields are populatedtest_proxy_admin_jwt_auth_handles_no_team_object— verifies None team_object is handled correctlymake test-unit)