Conversation
…_support…" This reverts commit 9661249.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryThis PR reverts PR #23816 ("docs: add v1.82.3 release notes") by deleting the release notes document and partially rolling back changes to Key observations:
Confidence Score: 3/5
|
| Filename | Overview |
|---|---|
| docs/my-website/release_notes/v1.82.3.md | Release notes file for v1.82.3 is deleted as part of the revert — straightforward deletion of the docs file added in PR #23816. |
| provider_endpoints_support.json | Several provider entries are modified: sagemaker_nova is replaced by charity_engine, nebius and zai endpoint support flags are reverted to false, serper URL is updated, and chutes/other providers gain new a2a/interactions fields. The removal of sagemaker_nova may leave the JSON out of sync with the codebase if the provider implementation remains. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[PR #23816: Add v1.82.3 release notes] -->|Adds| B[docs/my-website/release_notes/v1.82.3.md]
A -->|Modifies| C[provider_endpoints_support.json\n+ sagemaker_nova\n- charity_engine\n+ nebius/zai capabilities]
D[PR #23817: Revert #23816] -->|Deletes| B
D -->|Partially reverts| C
E[Codebase] -->|Still contains| F[sagemaker_nova provider impl\nPR #21542]
E -->|Still contains| G[nebius messages/responses support]
C -->|After revert| H[provider_endpoints_support.json\n- sagemaker_nova missing\n- nebius/zai capabilities = false]
F -.->|Code-JSON mismatch| H
G -.->|Code-JSON mismatch| H
Comments Outside Diff (2)
-
provider_endpoints_support.json, line 2573-2584 (link)sagemaker_novaremoved,charity_enginesubstitutedThe
sagemaker_novaprovider entry is being removed from this JSON and replaced withcharity_engine. However, the release notes that are being reverted (PR docs: add v1.82.3 release notes #23816) explicitly describesagemaker_novaas a new provider implementation added via PR feat: add sagemaker_nova provider for Amazon Nova models on SageMaker #21542. That underlying provider code is not being reverted by this PR — only the docs are. This means the codebase will still contain asagemaker_novaprovider implementation with no corresponding entry inprovider_endpoints_support.json, leaving the endpoint support table permanently out of sync with what the code actually supports. -
provider_endpoints_support.json, line 1486-1510 (link)Nebius and ZAI capabilities reverted to
falsewhile code may still support themmessages,responses, anda2afor bothnebiusandzaiare being set back tofalse(andinteractions: trueremoved). If the underlying implementation changes from PR docs: add v1.82.3 release notes #23816 (and referenced PRs such as feat: add sagemaker_nova provider for Amazon Nova models on SageMaker #21542) are still present in the codebase, this JSON will misrepresent those providers as not supporting these endpoints. The same concern applies tozaiat the equivalent block below.Consider whether a follow-up PR is needed to re-add the correct capability flags once the docs are ready to go out, or to clearly revert the implementation code as well.
Last reviewed commit: 6d4ffda
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6d4ffdad7d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| "sagemaker_nova": { | ||
| "display_name": "AWS SageMaker Nova (`sagemaker_nova`)", | ||
| "url": "https://docs.litellm.ai/docs/providers/aws_sagemaker", | ||
| "charity_engine": { |
There was a problem hiding this comment.
Restore unique provider key for SageMaker Nova
This block reuses the charity_engine key even though providers.charity_engine is already defined earlier in the same JSON object, so JSON parsing keeps only the later value and effectively removes the prior entry while also dropping the sagemaker_nova provider mapping that existed before this revert. That makes provider capability metadata inconsistent with supported providers (for example, sagemaker_nova is still a valid provider in litellm/constants.py) and causes downstream docs/catalog consumers of this JSON to miss SageMaker Nova endpoint support.
Useful? React with 👍 / 👎.
…3382) * fix: langfuse trace leak key on model params * fix: pop sensitive keys from langfuse * fixes * fix: set oauth2_flow when building MCPServer in _execute_with_mcp_client * fix: add oauth2_flow to NewMCPServerRequest and guard auto-detect with token_url * fix: narrow oauth2_flow type to Literal in NewMCPServerRequest * fix: align DefaultInternalUserParams Pydantic default with runtime fallback The Pydantic default for user_role was INTERNAL_USER, but all runtime provisioning paths (SSO, SCIM, JWT) fall back to INTERNAL_USER_VIEW_ONLY when no settings are saved. This caused the UI to show "Internal User" on fresh instances while new users actually got "Internal Viewer". * test: add regression test for fresh-instance default role sync Asserts that GET /get/internal_user_settings returns INTERNAL_USER_VIEW_ONLY on a fresh DB with no saved settings, matching the runtime fallback in SSO/SCIM/JWT provisioning. * Update tests/test_litellm/proxy/ui_crud_endpoints/test_proxy_setting_endpoints.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * Add unit tests for 5 previously untested UI dashboard files Tests added for: UiLoadingSpinner, HashicorpVaultEmptyPlaceholder, PageVisibilitySettings, errorUtils, and mcpToolCrudClassification. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: remove skip decorators from m2m tests now that oauth2_flow is set * [Fix] Privilege escalation: restrict /key/block, /key/unblock, and max_budget updates to admins Non-admin users (INTERNAL_USER) could call /key/block and /key/unblock on arbitrary keys, and modify max_budget on their own keys via /key/update. These endpoints are now restricted to proxy admins, team admins, or org admins. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore(ui): migrate DefaultUserSettings buttons from Tremor to antd * [Infra] Merging RC Branch with Main (#23786) * fix(test): add missing mocks for test_streamable_http_mcp_handler_mock The test was missing mocks for extract_mcp_auth_context and set_auth_context, causing the handler to fail silently in the except block instead of reaching session_manager.handle_request. This mirrors the fix already applied to the sibling test_sse_mcp_handler_mock. Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix(ci): route OpenAI models through chat completions in pass-through tests The test_anthropic_messages_openai_model_streaming_cost_injection test fails because the OpenAI Responses API returns 400 for requests routed through the Anthropic Messages endpoint. Setting LITELLM_USE_CHAT_COMPLETIONS_URL_FOR_ANTHROPIC_MESSAGES=true routes OpenAI models through the stable chat completions path instead. Cost injection still works since it happens at the proxy level. Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix(ci): fix assemblyai custom auth and router wildcard test flakiness 1. custom_auth_basic.py: Add user_role='proxy_admin' so the custom auth user can access management endpoints like /key/generate. The test test_assemblyai_transcribe_with_non_admin_key was hidden behind an earlier -x failure and was never reached before. 2. test_router_utils.py: Add flaky(retries=3) and increase sleep from 1s to 2s for test_router_get_model_group_usage_wildcard_routes. The async callback needs time to write usage to cache, and 1s is insufficient on slower CI hardware. Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * ci: retrigger CI pipeline Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix(mypy): use LitellmUserRoles enum instead of raw string in custom_auth_basic Fixes mypy error: Argument 'user_role' has incompatible type 'str'; expected 'LitellmUserRoles | None' Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> * fix: don't close HTTP/SDK clients on LLMClientCache eviction (#22926) * fix: don't close HTTP/SDK clients on LLMClientCache eviction Removing the _remove_key override that eagerly called aclose()/close() on evicted clients. Evicted clients may still be held by in-flight streaming requests; closing them causes: RuntimeError: Cannot send a request, as the client has been closed. This is a regression from commit fb72979. Clients that are no longer referenced will be garbage-collected naturally. Explicit shutdown cleanup happens via close_litellm_async_clients(). Fixes production crashes after the 1-hour cache TTL expires. * test: update LLMClientCache unit tests for no-close-on-eviction behavior Flip the assertions: evicted clients must NOT be closed. Replace test_remove_key_closes_async_client → test_remove_key_does_not_close_async_client and equivalents for sync/eviction paths. Add test_remove_key_removes_plain_values for non-client cache entries. Remove test_background_tasks_cleaned_up_after_completion (no more _background_tasks). Remove test_remove_key_no_event_loop variant that depended on old behavior. * test: add e2e tests for OpenAI SDK client surviving cache eviction Add two new e2e tests using real AsyncOpenAI clients: - test_evicted_openai_sdk_client_stays_usable: verifies size-based eviction doesn't close the client - test_ttl_expired_openai_sdk_client_stays_usable: verifies TTL expiry eviction doesn't close the client Both tests sleep after eviction so any create_task()-based close would have time to run, making the regression detectable. Also expand the module docstring to explain why the sleep is required. * docs(AGENTS.md): add rule — never close HTTP/SDK clients on cache eviction * docs(CLAUDE.md): add HTTP client cache safety guideline * [Fix] Install bsdmainutils for column command in security scans The security_scans.sh script uses `column` to format vulnerability output, but the package wasn't installed in the CI environment. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: handle string callback values in prometheus multiproc setup When callbacks are configured as a plain string (e.g., `callbacks: "my_callback"`) instead of a list, the proxy crashes on startup with: TypeError: can only concatenate str (not "list") to str Normalize each callback setting to a list before concatenating. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * bump: version 1.82.2 → 1.82.3 * fix(test): update test_startup_fails_when_db_setup_fails for opt-in enforcement The --enforce_prisma_migration_check flag is now required to trigger sys.exit(1) on DB migration failure, after #23675 flipped the default behavior to warn-and-continue. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(cost_calculator): use model name for per-request custom pricing when router_model_id has no pricing When custom pricing is passed as per-request kwargs (input_cost_per_token/output_cost_per_token), completion() registers pricing under the model name, but _select_model_name_for_cost_calc was selecting the router deployment hash (which has no pricing data), causing response_cost to be 0.0. Now checks whether the router_model_id entry actually has pricing before preferring it. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * Update litellm/proxy/management_endpoints/key_management_endpoints.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * fix: clear oauth2_flow when client_credentials set without token_url * chore(ui): use antd danger prop instead of tailwind for Remove button * feat: fetch blog posts from docs RSS feed instead of static JSON on GitHub * fix: remove unused Any import from get_blog_posts * [Fix] UI - Logs: Fix empty filter results showing stale data Remove `.length > 0` check so that when a backend filter returns an empty result set the table correctly shows no data instead of falling back to the previous unfiltered logs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * [Fix] Reapply empty filter fix after merge with main Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * [Fix] Prevent internal users from creating invalid keys via key/generate and key/update Internal users could exploit key/generate and key/update to create unbound keys (no user_id, no budget) or attach keys to non-existent teams. This adds validation for non-admin callers: auto-assign user_id on generate, reject invalid team_ids, and prevent removing user_id on update. Closes LIT-1884 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * [Fix] Remove duplicate get_team_object call in _validate_update_key_data Move the non-admin team validation into the existing get_team_object call site to avoid an extra DB round-trip. The existing call already fetches the team for limits checking — we now add the LIT-1884 guard there when team_obj is None for non-admin callers. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * [Fix] Skip key_alias re-validation on update/regenerate when alias unchanged When updating or regenerating a key without changing its key_alias, the existing alias was being re-validated against current format rules. This caused keys with legacy aliases (created before stricter validation) to become uneditable. Now validation only runs when the alias actually changes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * [Fix] Update log filter test to match empty-result behavior The test expected fallback to all logs when backend filters return empty, but the source was intentionally changed to show empty results instead of stale data. Updated test to match. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * [Feature] Disable custom API key values via UI setting Add disable_custom_api_keys UI setting that prevents users from specifying custom key values during key generation and regeneration. When enabled, all keys must be auto-generated, eliminating the risk of key hash collisions in multi-tenant environments. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * [Fix] Add disable_custom_api_keys to UISettings Pydantic model Without this field on the model, GET /get/ui_settings omits the setting from the response and field_schema, preventing the UI from reading it. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: Register DynamoAI guardrail initializer and enum entry (#23752) * fix: Register DynamoAI guardrail initializer and enum entry Fix the "Unsupported guardrail: dynamoai" error by: 1. Adding DYNAMOAI to SupportedGuardrailIntegrations enum 2. Implementing initialize_guardrail() and registries in dynamoai/__init__.py The DynamoAI guardrail was added in PR #15920 but never properly registered in the initialization system. The __init__.py was missing the guardrail_initializer_registry and guardrail_class_registry dictionaries that the dynamic discovery mechanism looks for at module load time. Fixes #22773 Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com> * Update litellm/proxy/guardrails/guardrail_hooks/dynamoai/__init__.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * Update litellm/proxy/guardrails/guardrail_hooks/dynamoai/__init__.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * test: Add tests for DynamoAI guardrail registration Verifies enum entry, initializer registry, class registry, instance creation, and global registry discovery. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * docs: add v1.82.3 release notes and update provider_endpoints_support.json (#23816) * [Feature] Add disable_custom_api_keys toggle to UI Settings page Adds a toggle switch to the admin UI Settings page so administrators can enable/disable custom API key values without making direct API calls. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Revert "docs: add v1.82.3 release notes and update provider_endpoints_support…" (#23817) This reverts commit 9661249. * [Fix] Rename toggle label to "Disable custom Virtual key values" Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * [Fix] Remove "API" from custom key description text Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(ui): CSV export empty on Global Usage page Aggregated endpoint returns empty breakdown.entities; fall back to grouping breakdown.api_keys by team_id. * Revert "fix: langfuse trace leak key on model params" * fix: support served_model_name for Baseten dedicated deployments Baseten dedicated deployments use an 8-char deployment ID for URL routing, but the vLLM server may expect a different model name in the request body (e.g. baseten-hosted/zai-org/GLM-5 vs wd1lndkw). Add served_model_name litellm_param to override the model field in the request body, and declare it in LiteLLMParamsTypedDict and GenericLiteLLMParams for IDE support. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Harshit Jain <harshitjain0562@gmail.com> Co-authored-by: Harshit Jain <48647625+Harshit28j@users.noreply.github.com> Co-authored-by: AlexKer <AlexKer@users.noreply.github.com> Co-authored-by: joereyna <joseph.reyna@gmail.com> Co-authored-by: Ryan Crabbe <rcrabbe@berkeley.edu> Co-authored-by: ryan-crabbe <128659760+ryan-crabbe@users.noreply.github.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com> Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* akto guardrails support in litellm * docs(guardrails): add akto to supported values in types/guardrails.py * frontend changes + fixes * feat(akto): update Akto guardrail integration with new configuration options and modes * docs(akto): enhance Akto documentation and configuration descriptions for clarity * feat(tests): add proxy server request headers to sample request data * refactor(akto): remove optional account and VXLAN IDs; update documentation and tests * feat(akto): add event_type parameter for enhanced observability in guardrail logging * refactor(akto): update environment variable references * refactor the python codes * refactor and fix linting * refactor(akto): remove unused event hook and clean up imports * refactor(akto): enhance AktoGuardrail with async support and improved logging * fix: Register DynamoAI guardrail initializer and enum entry (#23752) * fix: Register DynamoAI guardrail initializer and enum entry Fix the "Unsupported guardrail: dynamoai" error by: 1. Adding DYNAMOAI to SupportedGuardrailIntegrations enum 2. Implementing initialize_guardrail() and registries in dynamoai/__init__.py The DynamoAI guardrail was added in PR #15920 but never properly registered in the initialization system. The __init__.py was missing the guardrail_initializer_registry and guardrail_class_registry dictionaries that the dynamic discovery mechanism looks for at module load time. Fixes #22773 Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com> * Update litellm/proxy/guardrails/guardrail_hooks/dynamoai/__init__.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * Update litellm/proxy/guardrails/guardrail_hooks/dynamoai/__init__.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * test: Add tests for DynamoAI guardrail registration Verifies enum entry, initializer registry, class registry, instance creation, and global registry discovery. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * docs: add v1.82.3 release notes and update provider_endpoints_support.json (#23816) * Revert "docs: add v1.82.3 release notes and update provider_endpoints_support…" (#23817) This reverts commit 9661249. * Refactor Akto guardrail configuration and tests; update UI description and tags * add account and vxlan ID parameters to Akto guardrail initialization; update Akto logo format * enhance Akto guardrail documentation and improve error handling for non-JSON responses * address greptile issues * fix: update payload handling to use 'data' instead of 'json' in AktoGuardrail and adjust tests accordingly --------- Co-authored-by: Harshit Jain <48647625+Harshit28j@users.noreply.github.com> Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> Co-authored-by: Joe Reyna <joseph.reyna@gmail.com> Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Reverts #23816