fix(proxy): fix master key rotation Prisma validation errors#21330
Conversation
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
…28j/litellm into fix/sso_PKCE_deployments
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
…ile_content directly
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
batch_cost_calculator only checked the global cost map, ignoring deployment-level custom pricing (input_cost_per_token_batches etc.). Add optional model_info param through the batch cost chain and pass it from CheckBatchCost.
The test_db_schema_migration.py test requires pytest-postgresql but it was missing from dependencies, causing import errors: ModuleNotFoundError: No module named 'pytest_postgresql' Added pytest-postgresql ^6.0.0 to dev dependencies to fix test collection errors in proxy_unit_tests. This is a pre-existing issue, not related to PR BerriAI#21277. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
…openai Add doc for OpenAI Agents SDK with LiteLLM
…14_20262 Litellm oss staging 02 14 20262
…key-grace-period fix: virutal key grace period from env/UI
…ents fix: SSO PKCE support fails in multi-pod Kubernetes deployments
Resolved poetry.lock conflict by regenerating with Poetry 2.3.2. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
…ql-dependency fix(deps): add pytest-postgresql for db schema migration tests
…g-test-parallel fix(test): replace caplog with custom handler for parallel execution
…gging-mock fix(test): correct async mock for video generation logging test
…ver.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
fix(test): add cleanup fixture and no_parallel mark for MCP tests
…eta_header Litellm anthropic doc beta header
…erriAI#21125) (BerriAI#21244) * Fix tool params reported as supported for models without function calling (BerriAI#21125) JSON-configured providers (e.g. PublicAI) inherited all OpenAI params including tools, tool_choice, function_call, and functions — even for models that don't support function calling. This caused an inconsistency where get_supported_openai_params included "tools" but supports_function_calling returned False. The fix checks supports_function_calling in the dynamic config's get_supported_openai_params and removes tool-related params when the model doesn't support it. Follows the same pattern used by OVHCloud and Fireworks AI providers. * Style: move verbose_logger to module-level import, remove redundant try/except Address review feedback from Greptile bot: - Move verbose_logger import to top-level (matches project convention) - Remove redundant try/except around supports_function_calling() since it already handles exceptions internally via _supports_factory()
…AI#21239) * fix: handle missing database url in append_query_params * Update litellm/proxy/proxy_cli.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
…iAI#21323) PR BerriAI#19809 changed stateless=True to stateless=False to enable progress notifications for MCP tool calls. This caused the mcp library to enforce mcp-session-id headers on all non-initialize requests, breaking MCP Inspector, curl, and any client without automatic session management. Revert to stateless=True to restore compatibility with all MCP clients. The progress notification code already handles missing sessions gracefully (defensive checks + try/except), so no other changes are needed. Fixes BerriAI#20242
…ories + go to next page (BerriAI#21223) * feat(ui/): allow viewing content filter categories on guardrail info * fix(add_guardrail_form.tsx): add validation check to prevent adding empty content filter guardrails * feat(ui/): improve ux around adding new content filter categories easy to skip adding a category, so make it a 1-click thing
_rotate_master_key() used jsonify_object() which converts Python dicts to JSON strings. Prisma's Python client rejects strings for Json-typed fields — it requires prisma.Json() wrappers or native dicts. This affected three code paths: - Model table (create_many): litellm_params and model_info converted to strings, plus created_at/updated_at were None (non-nullable DateTime) - Config table (update): param_value converted to string - Credentials table (update): credential_values/credential_info converted to strings Fix: replace jsonify_object() with model_dump(exclude_none=True) + prisma.Json() wrappers for all Json fields. Wrap model delete+insert in a Prisma transaction for atomicity. Add try/except around MCP server rotation to prevent non-critical failures from blocking the entire rotation.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryFixes Prisma validation errors during master key rotation by replacing
Confidence Score: 4/5
|
| Filename | Overview |
|---|---|
| litellm/proxy/management_endpoints/key_management_endpoints.py | Correctly replaces jsonify_object() with model_dump(exclude_none=True) + prisma.Json() for all Json-typed fields in model, config, and credentials tables. Adds transaction wrapping for model delete+create. Wraps MCP rotation in try/except but doesn't fix the underlying serialization issue in the MCP path. |
| tests/test_litellm/proxy/management_endpoints/test_key_management_endpoints.py | Well-structured regression test using mocks (no real network calls). Validates that created_at/updated_at are excluded, Json fields are wrapped with prisma.Json(), and delete+create happen within a transaction. Exercises the real _add_model_to_db code path. |
Sequence Diagram
sequenceDiagram
participant Caller as _rotate_master_key
participant ProxyConfig as proxy_config
participant AddModel as _add_model_to_db
participant Prisma as Prisma DB
participant MCP as MCP rotation
Caller->>Prisma: find_many() models
Caller->>ProxyConfig: decrypt_model_list_from_db()
loop For each decrypted model
Caller->>AddModel: _add_model_to_db(should_create_model_in_db=False)
AddModel-->>Caller: LiteLLM_ProxyModelTable (in-memory)
Note over Caller: model_dump(exclude_none=True)<br/>+ prisma.Json() wrappers
end
rect rgb(200, 230, 200)
Note over Caller,Prisma: Transaction
Caller->>Prisma: delete_many() models
Caller->>Prisma: create_many() re-encrypted models
end
Caller->>Prisma: find_many() config
Caller->>ProxyConfig: decrypt & re-encrypt env vars
Caller->>Prisma: update() config with prisma.Json()
rect rgb(255, 230, 200)
Note over Caller,MCP: try/except (failure logged)
Caller->>MCP: rotate_mcp_server_credentials()
end
Caller->>Prisma: find_many() credentials
loop For each credential
Caller->>ProxyConfig: decrypt & re-encrypt credential
Note over Caller: model_dump(exclude_none=True)<br/>+ prisma.Json() wrappers
Caller->>Prisma: update() credential
end
Last reviewed commit: 1daea57
| try: | ||
| await rotate_mcp_server_credentials_master_key( | ||
| prisma_client=prisma_client, | ||
| touched_by=user_api_key_dict.user_id or LITELLM_PROXY_ADMIN_NAME, | ||
| new_master_key=new_master_key, | ||
| ) | ||
| except Exception as e: | ||
| verbose_proxy_logger.warning( | ||
| "Failed to rotate MCP server credentials: %s", str(e) | ||
| ) |
There was a problem hiding this comment.
Silent MCP rotation failure risks stale credentials
The MCP server rotation in rotate_mcp_server_credentials_master_key uses safe_dumps() (which returns a JSON string) for the credentials field, which is Json? in the Prisma schema. This is the same type of issue being fixed elsewhere in this PR (passing strings instead of prisma.Json() for Json-typed fields).
While wrapping in try/except is a reasonable defensive measure to prevent the entire rotation from failing, this means MCP server credentials will silently remain encrypted with the old master key after rotation. Subsequent operations using those credentials will fail because the proxy will try to decrypt them with the new master key.
Consider either:
- Fixing
rotate_mcp_server_credentials_master_keyto useprisma.Json()instead ofsafe_dumps()(similar to the fixes applied to the other tables in this PR), or - At minimum, re-raising the error or returning a status so the caller knows MCP rotation failed and can report it to the admin.
6edbeaa
into
BerriAI:litellm_oss_staging_02_16_2026
_rotate_master_key() used jsonify_object() which converts Python dicts to JSON strings. Prisma's Python client rejects strings for Json-typed fields — it requires prisma.Json() wrappers or native dicts.
This affected three code paths:
Fix: replace jsonify_object() with model_dump(exclude_none=True) + prisma.Json() wrappers for all Json fields. Wrap model delete+insert in a Prisma transaction for atomicity. Add try/except around MCP server rotation to prevent non-critical failures from blocking the entire rotation.
Relevant issues
Pre-Submission checklist
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewCI (LiteLLM team)
Type
🐛 Bug Fix
Changes
litellm/proxy/management_endpoints/key_management_endpoints.pyModel table serialization — Replace
jsonify_object(new_model.model_dump())withnew_model.model_dump(exclude_none=True)+prisma.Json()wrappers forlitellm_paramsandmodel_info. This fixes two issues:exclude_none=Truestripscreated_at: None/updated_at: None(letting Prisma@default(now())apply), andprisma.Json()wraps dicts socreate_many()accepts them forJsonfields.Model table atomicity — Wrap the delete + insert in
async with prisma_client.db.tx()so the operation is atomic (prevents model loss on failure).Config table serialization — Replace
jsonify_object(encrypted_env_vars)withprisma.Json(encrypted_env_vars)for theparam_valueJson field.Credentials table serialization — Replace
jsonify_object(encrypted_cred.model_dump())withmodel_dump(exclude_none=True)+prisma.Json()wrappers forcredential_valuesandcredential_info.MCP rotation error handling — Wrap
rotate_mcp_server_credentials_master_key()in try/except so MCP table issues (e.g., missing columns during migration) don't block the entire rotation.tests/test_litellm/proxy/management_endpoints/test_key_management_endpoints.pyAdded
test_rotate_master_key_model_data_valid_for_prisma— regression test verifying:created_at/updated_atare excluded from model datalitellm_params/model_infoare wrapped withprisma.Json()delete_manyis called inside the transaction