Skip to content

fix(proxy): fix master key rotation Prisma validation errors#21330

Merged
krrishdholakia merged 79 commits intoBerriAI:litellm_oss_staging_02_16_2026from
michelligabriele:fix/master-key-rotation-prisma-json
Feb 16, 2026
Merged

fix(proxy): fix master key rotation Prisma validation errors#21330
krrishdholakia merged 79 commits intoBerriAI:litellm_oss_staging_02_16_2026from
michelligabriele:fix/master-key-rotation-prisma-json

Conversation

@michelligabriele
Copy link
Collaborator

_rotate_master_key() used jsonify_object() which converts Python dicts to JSON strings. Prisma's Python client rejects strings for Json-typed fields — it requires prisma.Json() wrappers or native dicts.

This affected three code paths:

  • Model table (create_many): litellm_params and model_info converted to strings, plus created_at/updated_at were None (non-nullable DateTime)
  • Config table (update): param_value converted to string
  • Credentials table (update): credential_values/credential_info converted to strings

Fix: replace jsonify_object() with model_dump(exclude_none=True) + prisma.Json() wrappers for all Json fields. Wrap model delete+insert in a Prisma transaction for atomicity. Add try/except around MCP server rotation to prevent non-critical failures from blocking the entire rotation.

Relevant issues

Pre-Submission checklist

  • I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

CI (LiteLLM team)

  • Branch creation CI run Link:
  • CI run for the last commit Link:
  • Merge / cherry-pick CI run Links:

Type

🐛 Bug Fix

Changes

litellm/proxy/management_endpoints/key_management_endpoints.py

  1. Model table serialization — Replace jsonify_object(new_model.model_dump()) with new_model.model_dump(exclude_none=True) + prisma.Json() wrappers for litellm_params and model_info. This fixes two issues: exclude_none=True strips created_at: None / updated_at: None (letting Prisma @default(now()) apply), and prisma.Json() wraps dicts so create_many() accepts them for Json fields.

  2. Model table atomicity — Wrap the delete + insert in async with prisma_client.db.tx() so the operation is atomic (prevents model loss on failure).

  3. Config table serialization — Replace jsonify_object(encrypted_env_vars) with prisma.Json(encrypted_env_vars) for the param_value Json field.

  4. Credentials table serialization — Replace jsonify_object(encrypted_cred.model_dump()) with model_dump(exclude_none=True) + prisma.Json() wrappers for credential_values and credential_info.

  5. MCP rotation error handling — Wrap rotate_mcp_server_credentials_master_key() in try/except so MCP table issues (e.g., missing columns during migration) don't block the entire rotation.

tests/test_litellm/proxy/management_endpoints/test_key_management_endpoints.py

Added test_rotate_master_key_model_data_valid_for_prisma — regression test verifying:

  • created_at/updated_at are excluded from model data
  • litellm_params/model_info are wrapped with prisma.Json()
  • delete_many is called inside the transaction

Harshit28j and others added 30 commits February 3, 2026 07:54
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
batch_cost_calculator only checked the global cost map, ignoring
deployment-level custom pricing (input_cost_per_token_batches etc.).
Add optional model_info param through the batch cost chain and pass
it from CheckBatchCost.
The test_db_schema_migration.py test requires pytest-postgresql but it was
missing from dependencies, causing import errors:

  ModuleNotFoundError: No module named 'pytest_postgresql'

Added pytest-postgresql ^6.0.0 to dev dependencies to fix test collection
errors in proxy_unit_tests.

This is a pre-existing issue, not related to PR BerriAI#21277.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Sameerlite and others added 23 commits February 16, 2026 19:03
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
…openai

Add doc for OpenAI Agents SDK with LiteLLM
…14_20262

Litellm oss staging 02 14 20262
…key-grace-period

fix: virutal key grace period from env/UI
…ents

fix: SSO PKCE support fails in multi-pod Kubernetes deployments
Resolved poetry.lock conflict by regenerating with Poetry 2.3.2.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
…ql-dependency

fix(deps): add pytest-postgresql for db schema migration tests
…g-test-parallel

fix(test): replace caplog with custom handler for parallel execution
…gging-mock

fix(test): correct async mock for video generation logging test
…ver.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
fix(test): add cleanup fixture and no_parallel mark for MCP tests
…eta_header

Litellm anthropic doc beta header
…erriAI#21125) (BerriAI#21244)

* Fix tool params reported as supported for models without function calling (BerriAI#21125)

JSON-configured providers (e.g. PublicAI) inherited all OpenAI params
including tools, tool_choice, function_call, and functions — even for
models that don't support function calling. This caused an inconsistency
where get_supported_openai_params included "tools" but
supports_function_calling returned False.

The fix checks supports_function_calling in the dynamic config's
get_supported_openai_params and removes tool-related params when the
model doesn't support it. Follows the same pattern used by OVHCloud
and Fireworks AI providers.

* Style: move verbose_logger to module-level import, remove redundant try/except

Address review feedback from Greptile bot:
- Move verbose_logger import to top-level (matches project convention)
- Remove redundant try/except around supports_function_calling() since it
  already handles exceptions internally via _supports_factory()
…AI#21239)

* fix: handle missing database url in append_query_params

* Update litellm/proxy/proxy_cli.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
…iAI#21323)

PR BerriAI#19809 changed stateless=True to stateless=False to enable progress
notifications for MCP tool calls. This caused the mcp library to enforce
mcp-session-id headers on all non-initialize requests, breaking MCP
Inspector, curl, and any client without automatic session management.

Revert to stateless=True to restore compatibility with all MCP clients.
The progress notification code already handles missing sessions gracefully
(defensive checks + try/except), so no other changes are needed.

Fixes BerriAI#20242
…ories + go to next page (BerriAI#21223)

* feat(ui/): allow viewing content filter categories on guardrail info

* fix(add_guardrail_form.tsx): add validation check to prevent adding empty content filter guardrails

* feat(ui/): improve ux around adding new content filter categories

easy to skip adding a category, so make it a 1-click thing
_rotate_master_key() used jsonify_object() which converts Python dicts
to JSON strings. Prisma's Python client rejects strings for Json-typed
fields — it requires prisma.Json() wrappers or native dicts.

This affected three code paths:
- Model table (create_many): litellm_params and model_info converted to
  strings, plus created_at/updated_at were None (non-nullable DateTime)
- Config table (update): param_value converted to string
- Credentials table (update): credential_values/credential_info
  converted to strings

Fix: replace jsonify_object() with model_dump(exclude_none=True) +
prisma.Json() wrappers for all Json fields. Wrap model delete+insert
in a Prisma transaction for atomicity. Add try/except around MCP
server rotation to prevent non-critical failures from blocking the
entire rotation.
@vercel
Copy link

vercel bot commented Feb 16, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Ready Ready Preview, Comment Feb 16, 2026 6:57pm

Request Review

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Feb 16, 2026

Greptile Summary

Fixes Prisma validation errors during master key rotation by replacing jsonify_object() (which converts dicts to JSON strings incompatible with Prisma's Python client) with model_dump(exclude_none=True) + prisma.Json() wrappers for all Json-typed fields across the model, config, and credentials tables. Also adds transaction wrapping for the model table delete+create operation and defensive error handling around MCP server credential rotation.

  • Model table: exclude_none=True strips created_at/updated_at (both None when constructed in-memory), letting Prisma's @default(now()) apply. prisma.Json() wraps litellm_params and model_info as required by create_many().
  • Config table: param_value now wrapped with prisma.Json() instead of jsonify_object().
  • Credentials table: credential_values and credential_info conditionally wrapped with prisma.Json().
  • Transaction: Model delete+create now wrapped in prisma_client.db.tx() for atomicity.
  • MCP rotation: Wrapped in try/except to prevent non-critical failures from blocking the entire rotation, though the underlying safe_dumps() serialization issue in the MCP path is not addressed.
  • Test: Regression test verifies the model table serialization produces valid Prisma-compatible data.

Confidence Score: 4/5

  • This PR is safe to merge — it fixes real Prisma validation errors with correct serialization patterns and adds transaction safety.
  • The fix correctly addresses the root cause (jsonify_object converting dicts to strings that Prisma rejects) across three code paths (models, config, credentials). The transaction wrapping for model delete+create is a good improvement. The regression test covers the most critical path. The one concern is the MCP rotation being silently swallowed rather than properly fixed, which could leave MCP credentials encrypted with the old key after rotation.
  • Pay attention to litellm/proxy/management_endpoints/key_management_endpoints.py lines 3137-3146 — the MCP rotation try/except may mask a real serialization issue similar to the ones being fixed in this PR.

Important Files Changed

Filename Overview
litellm/proxy/management_endpoints/key_management_endpoints.py Correctly replaces jsonify_object() with model_dump(exclude_none=True) + prisma.Json() for all Json-typed fields in model, config, and credentials tables. Adds transaction wrapping for model delete+create. Wraps MCP rotation in try/except but doesn't fix the underlying serialization issue in the MCP path.
tests/test_litellm/proxy/management_endpoints/test_key_management_endpoints.py Well-structured regression test using mocks (no real network calls). Validates that created_at/updated_at are excluded, Json fields are wrapped with prisma.Json(), and delete+create happen within a transaction. Exercises the real _add_model_to_db code path.

Sequence Diagram

sequenceDiagram
    participant Caller as _rotate_master_key
    participant ProxyConfig as proxy_config
    participant AddModel as _add_model_to_db
    participant Prisma as Prisma DB
    participant MCP as MCP rotation

    Caller->>Prisma: find_many() models
    Caller->>ProxyConfig: decrypt_model_list_from_db()
    loop For each decrypted model
        Caller->>AddModel: _add_model_to_db(should_create_model_in_db=False)
        AddModel-->>Caller: LiteLLM_ProxyModelTable (in-memory)
        Note over Caller: model_dump(exclude_none=True)<br/>+ prisma.Json() wrappers
    end
    rect rgb(200, 230, 200)
        Note over Caller,Prisma: Transaction
        Caller->>Prisma: delete_many() models
        Caller->>Prisma: create_many() re-encrypted models
    end
    Caller->>Prisma: find_many() config
    Caller->>ProxyConfig: decrypt & re-encrypt env vars
    Caller->>Prisma: update() config with prisma.Json()
    rect rgb(255, 230, 200)
        Note over Caller,MCP: try/except (failure logged)
        Caller->>MCP: rotate_mcp_server_credentials()
    end
    Caller->>Prisma: find_many() credentials
    loop For each credential
        Caller->>ProxyConfig: decrypt & re-encrypt credential
        Note over Caller: model_dump(exclude_none=True)<br/>+ prisma.Json() wrappers
        Caller->>Prisma: update() credential
    end
Loading

Last reviewed commit: 1daea57

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

Comment on lines +3137 to +3146
try:
await rotate_mcp_server_credentials_master_key(
prisma_client=prisma_client,
touched_by=user_api_key_dict.user_id or LITELLM_PROXY_ADMIN_NAME,
new_master_key=new_master_key,
)
except Exception as e:
verbose_proxy_logger.warning(
"Failed to rotate MCP server credentials: %s", str(e)
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Silent MCP rotation failure risks stale credentials

The MCP server rotation in rotate_mcp_server_credentials_master_key uses safe_dumps() (which returns a JSON string) for the credentials field, which is Json? in the Prisma schema. This is the same type of issue being fixed elsewhere in this PR (passing strings instead of prisma.Json() for Json-typed fields).

While wrapping in try/except is a reasonable defensive measure to prevent the entire rotation from failing, this means MCP server credentials will silently remain encrypted with the old master key after rotation. Subsequent operations using those credentials will fail because the proxy will try to decrypt them with the new master key.

Consider either:

  1. Fixing rotate_mcp_server_credentials_master_key to use prisma.Json() instead of safe_dumps() (similar to the fixes applied to the other tables in this PR), or
  2. At minimum, re-raising the error or returning a status so the caller knows MCP rotation failed and can report it to the admin.

@krrishdholakia krrishdholakia changed the base branch from main to litellm_oss_staging_02_16_2026 February 16, 2026 23:12
@krrishdholakia krrishdholakia merged commit 6edbeaa into BerriAI:litellm_oss_staging_02_16_2026 Feb 16, 2026
10 of 18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.