Skip to content

🔄 Upstream Sync: LiteLLM v1.83.3-stable - #104

Merged
mateo-di merged 4859 commits into
carto/mainfrom
upstream-sync/v1.83.3-stable
Apr 20, 2026
Merged

🔄 Upstream Sync: LiteLLM v1.83.3-stable#104
mateo-di merged 4859 commits into
carto/mainfrom
upstream-sync/v1.83.3-stable

Conversation

@Cartofante

@Cartofante Cartofante commented Apr 14, 2026

Copy link
Copy Markdown
Collaborator

🔄 Upstream Sync: LiteLLM v1.83.3-stable

Syncs CARTO's LiteLLM fork with upstream stable release v1.83.3-stable.

Metric Value
Version 1.81.9v1.83.3-stable
Commits 4858
Files Changed 4477
Upstream Release v1.83.3-stable

Caution

⚠️ DO NOT SQUASH MERGE THIS PR

Use "Create a merge commit" only. Squashing destroys upstream history and breaks future syncs.


🧪 Pre-Merge Checklist

  • CI checks pass (lint, tests, Docker build)
  • CARTO customizations preserved
  • pyproject.toml version matches upstream

📊 Release Information (click to expand)
🔀 Branch Flow (click to expand)
  1. BerriAI/litellm:main merged into CartoDB/litellm:main
  2. ✅ Created dedicated sync branch: upstream-sync/v1.83.3-stable
  3. 📝 This PR: upstream-sync/v1.83.3-stablecarto/main

[!NOTE]
Why a dedicated branch? Allows pushing conflict resolution commits directly to this PR.

📝 CARTO-Specific File Guidelines (click to expand)

When reviewing or resolving conflicts:

✅ Keep CARTO Versions (Ours)

  • .github/workflows/carto_*.yaml - CARTO workflows
  • .github/workflows/carto-*.yml - CARTO workflows
  • CARTO_*.md, docs/CARTO_*.md - CARTO documentation

🔄 Accept Upstream (Theirs)

  • pyproject.toml - Version field
  • litellm/ - Core library code
  • tests/ - Upstream tests
  • requirements.txt - Dependencies

⚠️ Manual Review Required

  • Dockerfile, docker/Dockerfile.non_root - CARTO customizations
  • Makefile - Check # CARTO: sections
🔧 Conflict Resolution (click to expand)

If this PR has conflicts:

Option 1: Automated (Recommended)

The carto-upstream-sync-resolver workflow triggers automatically.

What it does:

  1. 🤖 Detects conflicts → 🔀 Merges carto/main → ✏️ Resolves conflicts → 🧪 Runs tests → 📌 Pushes to this PR

You just need to: Wait for resolution commits, verify CARTO customizations, merge.

[!TIP]
Single PR workflow! No separate resolution PR needed.

Option 2: Manual Resolution

git fetch origin
git checkout upstream-sync/v1.83.3-stable
git merge origin/carto/main  # Creates conflicts
# ... resolve conflicts ...
make lint && make test-unit
git push origin upstream-sync/v1.83.3-stable
📚 Documentation Links (click to expand)

🤖 This PR was automatically created by the carto-upstream-sync workflow.

CARTO Customizations Analysis

Overall Assessment: ✅ PASS

Decision Count Description
Upstream Substitutes 0 Upstream provides equivalent functionality
Customized Upstream 1 Upstream enhanced with CARTO-specific behavior
Preserved CARTO 15 Full CARTO implementation kept
Incorrectly Dropped 0 CARTO feature lost (needs fixing!)
Total 16

CARTO Feature Preservation Analysis

Summary

Decision Count
Upstream Substitutes 0
Customized Upstream 1
Preserved CARTO 15
Incorrectly Dropped 0

Overall Assessment: PASS

All 16 analyzed CARTO features have been correctly preserved in the upstream sync resolution.
The conflict resolution followed the CARTO-First strategy, maintaining all critical customizations
while accepting beneficial upstream changes where appropriate.

Feature Details

Preserved CARTO (15 features)

These features were fully preserved because upstream does not provide equivalent functionality:

PR Feature Category Verification
#70 Azure URL suffix stripping Azure select_azure_base_url_or_endpoint() with regex present
#69 Databricks empty delta handling Databricks Safe .get('content') access present
#68 OCI Gemini tool calling OCI uuid.uuid4().hex and _handle_tool_call_delta present
#66 Responses SessionHandler fix Responses API ResponsesSessionHandler import present
#61 Azure Responses API URL Responses API _get_base_azure_url usage present
#58 Snowflake tool calling Responses API _transform_tool_calls_from_snowflake_to_openai present
#54 JSON repair for streaming Streaming _validate_and_repair_tool_arguments present
#38 Snowflake PAT auth Streaming PAT token detection present
#7 APScheduler memory fix Core replace_existing, coalesce settings present
#5 Redis session patch Responses API CARTO PATCH comment present
#94 .gitignore .worktrees/ Core Entry present
#91 Features manifest CI CI/CD carto-features.yml and workflow present
#90 Multi-arch Docker builds Infrastructure Multi-arch workflow present
#76 CARTO-First strategy Core CARTO_CLAUDE.md (1148 lines) preserved
#13 Prisma offline binaries Infrastructure Build-time download logic present

Customized Upstream (1 feature)

These features combine CARTO customizations with upstream improvements:

PR Feature Resolution
#55 wolfi-base Docker image CARTO's cgr.dev/chainguard/wolfi-base preserved with upstream's SHA256 pinning

Upstream Substitutes (0 features)

No CARTO features were replaced by equivalent upstream functionality.

Issues Found

None - All CARTO features were correctly preserved.

Manifest Verification

All patterns from .github/carto-features.yml were verified:

  • OCI Gemini Tool Call UUIDs: uuid.uuid4().hex present
  • Snowflake Streaming + Tool Calling: SnowflakeStreamingHandler, _transform_messages present
  • Snowflake Full URL Passthrough: CARTO: skip path construction comment present
  • Azure URL Suffix Stripping: regex pattern present

CI/CD Workflows

All 12 CARTO-specific workflows are present:

  • carto-features-check.yml
  • carto-ghcr-deploy.yaml
  • carto-n8n-notifier.yml
  • carto-release.yaml
  • carto-schema-sync-validator.yml
  • carto-slack-changelog.yml
  • carto-upstream-sync-ci-fixer.yml
  • carto-upstream-sync-conflict-detector.yml
  • carto-upstream-sync-customizations-analyzer.yml
  • carto-upstream-sync-main.yml
  • carto-upstream-sync-ready-checker.yml
  • carto-upstream-sync-resolver.yml

Conclusion

The upstream sync resolution for PR #104 correctly preserved all CARTO customizations.
The CARTO-First resolution strategy was properly followed, and no features were incorrectly dropped.


Feature-by-Feature Breakdown

PR #70: fix(azure): Strip operation suffixes from deployment URLs

  • Decision: 🔒 Preserved CARTO (high confidence)
  • Reason: CARTO's select_azure_base_url_or_endpoint() function with regex to strip /chat/completions, /embeddings, etc. is fully preserved in resolved. Upstream does not provide equivalent functionality.
  • Files: litellm/llms/azure/common_utils.py, tests/test_litellm/llms/azure/test_azure_common_utils.py
  • Recommendation: Correct decision - CARTO functionality preserved with tests

PR #69: fix(databricks): Handle empty delta in GPT-5 streaming

  • Decision: 🔒 Preserved CARTO (high confidence)
  • Reason: Safe .get('content') access pattern preserved in Databricks transformation to handle empty delta chunks from GPT-5 models.
  • Files: litellm/llms/databricks/chat/transformation.py
  • Recommendation: Correct decision - prevents streaming errors with GPT-5

PR #68: fix(oci): Add tool calling support for OCI Gemini streaming

  • Decision: 🔒 Preserved CARTO (high confidence)
  • Reason: OCI tool calling with uuid.uuid4().hex for tool call IDs and _handle_tool_call_delta handler both verified present.
  • Files: litellm/llms/oci/chat/transformation.py, litellm/responses/litellm_completion_transformation/streaming_iterator.py
  • Recommendation: Correct decision - enables OCI Gemini tool calling

PR #66: fix(responses): Fix _ENTERPRISE_ResponsesSessionHandler NameError

  • Decision: 🔒 Preserved CARTO (high confidence)
  • Reason: ResponsesSessionHandler import and usage preserved in transformation.py. The session handler integration is maintained.
  • Files: litellm/responses/litellm_completion_transformation/transformation.py
  • Recommendation: Correct decision - session handler works correctly

PR #61: fix: Azure Responses API URL construction

  • Decision: 🔒 Preserved CARTO (high confidence)
  • Reason: Azure Responses API URL construction with _get_base_azure_url preserved for handling deployment paths correctly.
  • Files: litellm/llms/azure/responses/transformation.py
  • Recommendation: Correct decision - Azure deployment URLs work correctly

PR #58: fix: Enable Snowflake tool calling via Responses API

  • Decision: 🔒 Preserved CARTO (high confidence)
  • Reason: Snowflake tool calling transformation with _transform_tool_calls_from_snowflake_to_openai and tool_choice_value check preserved.
  • Files: litellm/llms/snowflake/chat/transformation.py, litellm/responses/litellm_completion_transformation/transformation.py
  • Recommendation: Correct decision - Snowflake tool calling works

PR #55: fix: switch Docker base image to wolfi-base

  • Decision: 🔧 Customized Upstream (high confidence)
  • Reason: CARTO's wolfi-base image preserved with upstream's SHA256 pinning enhancement. Resolved uses cgr.dev/chainguard/wolfi-base@sha256:... combining both improvements.
  • Files: docker/Dockerfile.non_root
  • Recommendation: Correct decision - best of both: secure base + pinned version

PR #54: fix: repair malformed JSON in streaming tool call arguments

  • Decision: 🔒 Preserved CARTO (high confidence)
  • Reason: _validate_and_repair_tool_arguments function fully preserved to fix malformed JSON from duplicate/overlapping chunks.
  • Files: litellm/litellm_core_utils/streaming_chunk_builder_utils.py, tests/test_litellm/litellm_core_utils/test_streaming_chunk_builder_utils.py
  • Recommendation: Correct decision - prevents streaming JSON errors

PR #38: fix: Snowflake PAT auth and Claude streaming support

  • Decision: 🔒 Preserved CARTO (high confidence)
  • Reason: Snowflake PAT token detection and SnowflakeStreamingHandler preserved in common_utils and transformation.
  • Files: litellm/llms/snowflake/common_utils.py, litellm/llms/snowflake/chat/transformation.py
  • Recommendation: Correct decision - Snowflake auth works correctly

PR #7: Configure scheduler with memory leak prevention settings

  • Decision: 🔒 Preserved CARTO (high confidence)
  • Reason: APScheduler memory leak fix settings (replace_existing, coalesce) preserved in proxy_server.py.
  • Files: litellm/proxy/proxy_server.py
  • Recommendation: Correct decision - prevents memory leaks

PR #5: fix: Responses API Redis session timing

  • Decision: 🔒 Preserved CARTO (high confidence)
  • Reason: CARTO PATCH comment and Redis session storage preserved to avoid batch processing delays.
  • Files: litellm/responses/litellm_completion_transformation/handler.py
  • Recommendation: Correct decision - Redis sessions work correctly

PR #94: chore(gitignore): add .worktrees/ to .gitignore

  • Decision: 🔒 Preserved CARTO (high confidence)
  • Reason: .gitignore includes .worktrees/ addition and other CARTO-specific patterns.
  • Files: .gitignore
  • Recommendation: Correct decision - supports git worktrees

PR #91: feat(.github): add CARTO features manifest CI verification

  • Decision: 🔒 Preserved CARTO (high confidence)
  • Reason: carto-features.yml manifest and carto-features-check.yml workflow preserved. All 12 CARTO workflows present.
  • Files: .github/carto-features.yml, .github/workflows/carto-features-check.yml
  • Recommendation: Correct decision - CI verification works

PR #90: feat(docker): multi-arch builds (AMD64 + ARM64)

  • Decision: 🔒 Preserved CARTO (high confidence)
  • Reason: Multi-arch Docker build workflow preserved in carto-ghcr-deploy.yaml and docker-build-multiarch.yaml.
  • Files: .github/workflows/carto-ghcr-deploy.yaml, .github/workflows/docker-build-multiarch.yaml
  • Recommendation: Correct decision - multi-arch builds work

PR #76: feat: Implement CARTO-First resolution strategy

  • Decision: 🔒 Preserved CARTO (high confidence)
  • Reason: CARTO_CLAUDE.md with 1148 lines fully preserved. Upstream version is empty (1 line).
  • Files: CARTO_CLAUDE.md
  • Recommendation: Correct decision - CARTO documentation preserved

PR #13: Download prisma binaries on build time

  • Decision: 🔒 Preserved CARTO (high confidence)
  • Reason: Prisma offline binaries support preserved in Dockerfile.non_root and litellm-proxy-extras.
  • Files: docker/Dockerfile.non_root, litellm-proxy-extras/litellm_proxy_extras/utils.py
  • Recommendation: Correct decision - offline Prisma works

Analyzed: 2026-04-14 18:56:53 UTC
Workflow Run: #20
Analysis Artifacts: Download JSON/MD
Method: Claude Code (Opus 4.5) post-resolution semantic analysis

yuneng-berri and others added 30 commits March 30, 2026 13:53
…gration tests

gemini-2.0-flash is no longer available to new users, causing 404 errors
in llm_translation_testing tests.
[Fix] Update gemini-2.0-flash to gemini-2.5-flash in test_gemini
[Fix] Router code coverage CI failure for health check filter tests
…lcini

[Fix] WatsonX Tests Failing on CI Due to Missing Env Vars
PR BerriAI#24755 renamed `azure_api_key_header` to `AZURE_AI_API_KEY_header` in
the test file but did not update the actual function signatures of
`get_api_key()` and `_user_api_key_auth_builder()`, causing TypeError
on all affected test cases.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…name

[Fix] Correct kwarg name in test_user_api_key_auth tests
…added

fix(responses): emit content_part.added event for non-OpenAI models
ryan-crabbe-berri and others added 17 commits April 4, 2026 16:11
…er membership check

Bug 1: internal users hit route-level 403 on /guardrails/submissions.
The route wasn't in self_managed_routes, so the route allowlist rejected
non-admin callers before our endpoint's team-scoping ran. Added
/guardrails/submissions and /guardrails/submissions/{guardrail_id} to
self_managed_routes.

Bug 2: register_guardrail 403'd non-admins registering for teams in
their user.teams list. It used get_team_membership() which reads the
litellm_teammembership join table, but that row is only created when
the team has a budget (management_helpers/utils.py:225). Switched to
the _get_user_team_ids helper (reads user_obj.teams), making it
consistent with list_guardrail_submissions.

UI: moved the Test Playground tab inside the isAdmin conditional in
guardrails.tsx. Internal users now see only the Submitted Guardrails
tab; admins still see all four.

Tests: added coverage for non-admin register paths (cross-team allowed
and cross-team forbidden).
…a-id

feat(ui): expose Azure Entra ID credential fields in provider form
…er-model-rate-limit-ui

feat(ui): add per-model rate limits to team edit/info views
fix(ui): use entity key for usage export display
… (BerriAI#25155)

* Litellm ishaan march23 - MCP Toolsets + GCP Caching fix  (BerriAI#25146)

* feat(mcp): MCP Toolsets — curated tool subsets from one or more MCP servers (BerriAI#24335)

* feat(mcp): add LiteLLM_MCPToolsetTable and mcp_toolsets to ObjectPermissionTable

* feat(mcp): add prisma migration for MCPToolset table

* feat(mcp): add MCPToolset Python types

* feat(mcp): add toolset_db.py with CRUD helpers for MCPToolset

* feat(mcp): add toolset CRUD endpoints to mcp_management_endpoints

* fix(mcp): skip allow_all_keys servers when explicit mcp_servers permission is set (toolset scope fix)

* feat(mcp): add _apply_toolset_scope and toolset route handling in server.py

* fix(mcp): resolve toolset names in responses API before fetching tools

* feat(mcp): add mcp_toolsets field to LiteLLM_ObjectPermissionTable type

* feat(mcp): register LiteLLM_MCPToolsetTable in prisma client initialization

* feat(mcp): validate mcp_toolsets in key-vs-team permission check

* feat(mcp): register toolset routes in proxy_server.py

* feat(mcp): add MCPToolset and MCPToolsetTool TypeScript types

* feat(mcp): add fetchMCPToolsets, createMCPToolset, updateMCPToolset, deleteMCPToolset API functions

* feat(mcp): add useMCPToolsets React Query hook

* feat(mcp): add toolsets (purple) as third option type in MCPServerSelector

* feat(mcp): extract toolsets from combined MCP field in key form

* feat(mcp): extract toolsets from combined MCP field in team form

* feat(mcp): show toolsets section in MCPServerPermissions read view

* feat(mcp): pass mcp_toolsets through object_permissions_view

* feat(mcp): add MCPToolsetsTab component for creating and managing toolsets

* feat(mcp): add Toolsets tab to mcp_servers.tsx

* feat(mcp): pass mcpToolsets to playground chat and responses API calls

* feat(mcp): generate correct server_url for toolsets in playground API calls

* docs(mcp): add MCP Toolsets documentation

* docs(mcp): add mcp_toolsets to sidebar

* fix(mcp): replace x-mcp-toolset-id header with ContextVar to prevent client forgery

* fix(mcp): use ContextVar + StreamingResponse for toolset MCP routes (fixes SSE streaming)

* fix(mcp): cache toolset permission lookups to avoid per-request DB calls

* test(mcp): add tests for toolset scope enforcement, ContextVar isolation, and access control

* fix(mcp): cache toolset name lookups in MCPServerManager to avoid per-request DB calls

* fix(mcp): prevent body_iter deadlock + use cached toolset lookup in responses API

- _stream_mcp_asgi_response: add done callback to handler_task that puts
  the EOF sentinel on body_queue when the task exits, preventing body_iter
  from hanging forever if the handler raises after headers are sent.
- litellm_proxy_mcp_handler: replace raw get_mcp_toolset_by_name() DB call
  with global_mcp_server_manager.get_toolset_by_name_cached() so toolset
  resolution uses the 60s TTL cache added for this purpose instead of
  hitting the DB on every responses-API request.

* fix(mcp): toolset access control, asyncio fix, and real unit tests

- server.py: _apply_toolset_scope now enforces that non-admin keys must
  have the requested toolset_id in their mcp_toolsets grant list;
  admin keys always bypass the check.
- mcp_management_endpoints.py: three access-control fixes:
  * fetch_mcp_toolsets: non-admin keys with mcp_toolsets=None now
    return [] instead of all toolsets (only admins get 'all' when
    the field is absent)
  * fetch_mcp_toolset: non-admin keys that haven't been granted the
    requested toolset_id now get 403 instead of the full result
  * add_mcp_toolset: duplicate toolset_name now returns 409 Conflict
    instead of an opaque 500
- proxy_server.py: use asyncio.get_running_loop() instead of
  get_event_loop() inside an already-running coroutine (Python 3.10+).
- test_mcp_toolset_scope.py: replace four hollow tests that only
  asserted local variable properties with real tests that call the
  production fetch_mcp_toolsets() and handle_streamable_http_mcp()
  functions with mocked dependencies.

* fix(mcp): add mcp_toolsets to ObjectPermissionBase, fix multi-toolset overwrite, fix delete 404, allow standalone key toolsets

* fix(mcp): add auth check on toolset resolution in responses API; union mcp_servers in _merge_toolset_permissions

* fix(mcp): handle RecordNotFoundError in update_mcp_toolset; union direct servers with toolset servers

* fix(mcp): use _user_has_admin_view; deny None mcp_toolsets for non-admin; use direct RecordNotFoundError import; fix docstring

* fix(mcp): add @default(now()) to MCPToolsetTable.updated_at; fix test for non-admin toolset access

* fix: use UniqueViolationError import; guard _ensure_eof for error/cancel only

* fix(mcp): preserve mcp_access_groups in toolset scope, use shared Redis cache for toolset perms

- Remove mcp_access_groups=[] from _apply_toolset_scope (server.py) and the
  responses API toolset path (litellm_proxy_mcp_handler.py). A key's access-group
  grants remain valid even when the request is scoped to a single toolset; clearing
  them silently revoked legitimate entitlements.

- Switch resolve_toolset_tool_permissions and get_toolset_by_name_cached to use
  user_api_key_cache (Redis-backed DualCache in production) instead of per-instance
  in-memory dicts. Cache entries are now shared across workers, eliminating the
  per-worker stale-toolset-permission window flagged as a P1 by Greptile.

- Use union merge (set union of tool names per server) when applying toolset
  permissions in the responses API path so direct-server tool restrictions are not
  overwritten by toolset permissions.

* fix(mcp): return 404 when edit_mcp_toolset target does not exist

* fix(mcp): align mcp_toolsets default to None in LiteLLM_ObjectPermissionTable

* fix(mcp): admin toolset visibility, in-place tool name mutation, test helper coercion

* fix(mcp): treat None/[] team mcp_toolsets as no restriction in key validation

* fix(mcp): allow_all_keys backward compat, blocked_tools API write-path, efficient startup query

* fix(mcp): use _mcp_active_toolset_id ContextVar to detect toolset scope, avoiding DB-default false-positive

* fix(mcp): remove dead toolset cache stubs, log invalidation failures, align schema updated_at defaults

* fix(mcp): deserialise MCPToolset from Redis cache hit, replace fastapi import in test

* fix(mcp): evict name-cache on toolset mutation, 409 on rename conflict, warning-level list errors

* fix(redis): regenerate GCP IAM token per connection for async cluster (BerriAI#24426)

* fix(redis): regenerate GCP IAM token per connection for async cluster clients

Async RedisCluster was generating the IAM token once at startup and
storing it as a static password. After the 1-hour GCP token TTL, any
new connection (including to newly-discovered cluster nodes) would fail
to authenticate.

Fix: introduce GCPIAMCredentialProvider that implements redis-py's
CredentialProvider protocol. It calls _generate_gcp_iam_access_token()
on every new connection, matching what the sync redis_connect_func
already does. async_redis.RedisCluster accepts a credential_provider
kwarg which is invoked per-connection.

* refactor(redis): move GCPIAMCredentialProvider to its own file

Extract GCPIAMCredentialProvider and _generate_gcp_iam_access_token
into litellm/_redis_credential_provider.py. _redis.py imports them
from there, keeping the public API unchanged.

* fix: address Greptile review issues

- GCPIAMCredentialProvider now inherits from redis.credentials.CredentialProvider
  so redis-py's async path calls get_credentials_async() properly
- move _redis_credential_provider import to top of _redis.py (PEP 8)
- remove dead else-branch that silently no-oped (gcp_service_account from
  redis_kwargs.get() was always None since it's popped by _get_redis_client_logic)
- remove mid-function 'from litellm import get_secret_str' inline import
- remove unused 'call' import from test_redis.py

* chore: retrigger CI/review

* chore: sync schema.prisma copies from root

* chore: sync schema.prisma copies from root

* fix(proxy_server): use bounded asyncio.Queue with maxsize to prevent unbounded growth

* fix(a2a/pydantic_ai): make api_base Optional to match base class signature

* fix(a2a/pydantic_ai): make api_base Optional in handler and guard against None

* fix(mcp): remove unused get_all_mcp_servers import

* fix(mcp): remove unused MCPToolset import

* refactor(mcp): extract toolset permission logic to reduce statement count below PLR0915 limit

* fix(tests): update reload_servers_from_database tests to mock prisma directly

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* fix(toolset_db): lazy-import prisma to avoid ImportError when prisma not installed

* fix(tests): update UI tests for toolset tab and updated empty state text

* fix(tests): add get_mcp_server_by_name to fake_manager stub

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
The test was hitting "No QueryClient set" because TeamGuardrailsTab
(which pulls in useRegisterGuardrail) was not mocked alongside the
other tab children. Added a mock.

Also: the "+ Add New Guardrail" assertion was silently relying on
Tremor Tabs rendering all panels at once. antd Tabs only renders
the active tab's content, and defaultActiveKey is "submitted", so
the button in the "Guardrails" tab wasn't in the DOM. Clicking the
Guardrails tab first before asserting.
* added support for metadata (BerriAI#24261)

* added support for metadata

* fix: PR review - meta truthiness, BlobResourceContents mimeType, add Blob+empty meta tests

Made-with: Cursor

* pyproject to .25

* feat(teams): resolve access group models/MCPs/agents in team endpoints

Add access_group_models, access_group_mcp_server_ids, and
access_group_agent_ids to /team/info and /v2/team/list responses.
These fields contain resources inherited from access groups, kept
separate from direct assignments so the UI can distinguish the source.

Backend: _resolve_access_group_resources() helper resolves access
group resources via existing _get_*_from_access_groups() functions.

UI: Teams table and detail view show direct models as blue badges
and access-group-sourced models as green badges.

* perf(teams): single-pass access group resolution + asyncio.gather in list endpoint

- Fetch each access group object once and extract all 3 resource fields
  in a single pass instead of 3 separate calls (3N → N lookups)
- Use asyncio.gather to resolve access groups across teams concurrently
  in list_team_v2 instead of sequential awaits
- Add 5 unit tests for _resolve_access_group_resources

* docs: add default_team_params to config reference and update examples

- Add default_team_params to litellm_settings reference table in
  config_settings.md with all sub-fields documented
- Update self_serve.md and msft_sso.md examples to include
  team_member_permissions, tpm_limit, and rpm_limit
- Fix misleading comment that implied default_team_params only applies
  to SSO auto-created teams — it applies to all /team/new calls

* docs: clarify that models sub-field only applies to SSO auto-created teams

* fix: lazy import get_access_object to break cyclic import + short-circuit all-proxy-models display

- Remove get_access_object from module-level import in team_endpoints.py
  and use a lazy _get_access_object wrapper to avoid cyclic dependency
- Add _prisma_client is None early-exit guard in _resolve_access_group_resources
- Short-circuit UI to show "All Proxy Models" when team.models is empty
  or contains "all-proxy-models", skipping access group model resolution

* add: making organizations a select instead of read only badges

* fix(ui): only send organization_id when changed and use raw initial value

* fix(ui): add paginated team search to usage page filter

Replace the static team dropdown on the usage page with a new
TeamMultiSelect component that uses the paginated v2/team/list
endpoint with debounced server-side search and infinite scroll.

* fix(ui): fix imports and update placeholder for team multi select

* fix(ui): wire team_id filter to key alias dropdown on Virtual Keys tab

The Key Alias dropdown on the Virtual Keys page was showing aliases from
all teams regardless of which team was selected. The team_id was never
passed through the frontend chain to the backend /key/aliases endpoint.

- Backend: add optional team_id query param to /key/aliases endpoint
- networking.tsx: add team_id param to keyAliasesCall
- useKeyAliases: accept and forward team_id to API call and query key
- filter.tsx: pass allFilters context to custom filter components
- PaginatedKeyAliasSelect: read Team ID from allFilters and pass to hook

* fix(tests): correct mock targets in TestResolveAccessGroupResources

Three tests were patching the non-existent `get_access_object` instead
of `_get_access_object` (the lazy-import wrapper), causing AttributeError.
Also added missing `prisma_client` mock so tests get past the early-exit
guard and actually exercise the resolution logic.

* fix: use direct attribute access with or [] fallback in _resolve_access_group_resources

Replace getattr(ag, "field", []) with ag.field or [] for cleaner
access and safe handling if a field is None.

* fix(ui): remove model source legend from team detail view

The blue/green color distinction is self-explanatory; the legend added
visual clutter without providing enough value.

* fix(ui): add missing access_group fields to TeamData.team_info type

The TeamData interface was missing access_group_models,
access_group_mcp_server_ids, and access_group_agent_ids fields,
causing a TypeScript build failure.

* perf(teams): batch-fetch access groups in single DB query

Replace per-ID _resolve_access_group_resources loop with a single
find_many call that deduplicates IDs across all teams. Removes the
N+1 query pattern on cold cache for the team list endpoint.

* refactor(proxy): extract helpers to fix PLR0915 violations

Extract `_apply_non_admin_alias_scope` from `key_aliases`,
`_resolve_team_access_group_resources` from `team_info`, and
`_enforce_list_team_v2_access` from `list_team_v2` to bring each
function under ruff's 50-statement limit. No behavior changes.

* test(ui): update tests to match new team_id / access-group signatures

- useKeyAliases, PaginatedKeyAliasSelect: add trailing `undefined` to
  spy matchers for the new `team_id` param on `useInfiniteKeyAliases`
  and `keyAliasesCall`.
- EntityUsage: mock new `TeamMultiSelect` child so QueryClientProvider
  is not required for team-entity tests.
- ModelsCell: replace the overflow-accordion test with one that
  verifies the new collapse-on-`all-proxy-models` behavior (no
  accordion, single badge).

* fix(ui): send null (not '') for cleared organization_id on team update

AntD <Select allowClear> returns undefined when the user clears the
selection. Coalescing to "" caused the team-update payload to carry
organization_id: "" instead of null, relying on the backend to coerce
it. Send null directly so the intent is explicit at the source.

* poetry

* chore: regen poetry.lock for litellm-proxy-extras 0.4.64 bump

* chore: update Next.js build artifacts (2026-04-04 17:55 UTC, node v22.16.0)

---------

Co-authored-by: shivam <shivam@uni.minerva.edu>
Co-authored-by: Ryan Crabbe <ryan@berri.ai>
Co-authored-by: yuneng-jiang <yuneng@berri.ai>

* Tag query fix (BerriAI#25094)

* feat(tag-spend): implement separate scheduler job for daily tag spend updates

* fix(docker): add g++ to build dependencies in Dockerfile

* initial test cases. TODO: check scheduler init and test cases in proxy_server related to it

* resolved QPS issue when redis transaction buffer is enabled

* resolving circular import error flagged by greptile

* fix(mypy): use Optional[str] for api_base in PydanticAI provider to match superclass signature

---------

Co-authored-by: Shivam Rawat <shivam@berri.ai>
Co-authored-by: shivam <shivam@uni.minerva.edu>
Co-authored-by: Ryan Crabbe <ryan@berri.ai>
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
Co-authored-by: Harish <harishgokul01@gmail.com>
Co-authored-by: Ishaan Jaffer <ishaan@berri.ai>
feat: allow adding team guardrails from the UI
* bump litellm-proxy-extras version to 0.4.65

* bump litellm-proxy-extras==0.4.65 in pyproject.toml

* bump litellm-proxy-extras==0.4.65 in requirements.txt
* bump litellm-enterprise version to 0.1.36

* bump litellm-enterprise==0.1.36 in pyproject.toml

* bump litellm-enterprise==0.1.36 in requirements.txt
Automatic sync from upstream BerriAI/litellm tag v1.83.3-stable

Strategy: Merge with history preservation (main syncs to stable tag)
@github-actions

Copy link
Copy Markdown

No description provided.

@Cartofante

Copy link
Copy Markdown
Collaborator Author

🤖 Conflict Resolution Started

Status: ⏳ In progress...

Claude Code (Opus 4.5) is resolving merge conflicts in this PR.

Step Status
🔍 Analyze conflicts In progress
✏️ Resolve files Pending
🧪 Run tests Pending
📌 Push to PR Pending

Note

This may take 30-90 minutes for large PRs. Resolution commits will be pushed directly to this PR.

📋 Resolution Process (click to expand)
  1. 🔍 Analyzing conflicts in all files
  2. 📋 Reading CARTO customization guidelines
  3. ⚖️ Applying resolution priorities (CARTO first, then upstream)
  4. ✏️ Resolving conflicts file by file
  5. 🧪 Running tests (lint, mypy, unit tests)
  6. 📌 Pushing resolution directly to this PR

View workflow run →

Conflicts resolved by Claude Code following CARTO priority rules.

Resolution strategy:
- Preserved CARTO customizations (workflows, docs, infrastructure)
- Accepted upstream improvements (core litellm, tests, dependencies)
- Manually merged mixed files (Dockerfile, Makefile)

This is a MERGE COMMIT with both main and carto/main as parents,
preserving full git history from upstream.

Resolves: #104
@Cartofante

Copy link
Copy Markdown
Collaborator Author

✅ Conflict Resolution Complete

All conflicts resolved and pushed to this PR.

Step Status
🔍 Analyze conflicts ✅ Complete
✏️ Resolve files ✅ Complete
🧪 Verify resolution ✅ Complete
📌 Push to PR ✅ Complete

Important

Ready to merge! Use "Create a merge commit" — do NOT squash or rebase.


CARTO Customization Decisions

Summary

Decision Count
Upstream Substitutes 0
Preserved CARTO 3
Merged/Customized 8
Synced (required) 3

Upstream Substitutes

Files where upstream now provides equivalent functionality:

  • None - all CARTO features are unique customizations

Preserved CARTO

Files where CARTO implementation was kept entirely:

Merged/Customized

Files where both sources were combined:

  • litellm/litellm_core_utils/streaming_chunk_builder_utils.py - Kept CARTO JSON repair function + upstream structure
  • litellm/llms/azure/common_utils.py - Kept CARTO URL suffix stripping + upstream NamedTuple import
  • litellm/llms/oci/chat/transformation.py - Kept CARTO uuid.uuid4().hex feature + upstream OCI signer support
  • litellm/llms/snowflake/chat/transformation.py - Kept CARTO tool_names parameter in _transform_tool_choice
  • litellm/responses/litellm_completion_transformation/transformation.py - Kept CARTO empty assistant message filter
  • tests/test_litellm/litellm_core_utils/test_streaming_chunk_builder_utils.py - Merged upstream tests + CARTO JSON repair tests
  • tests/test_litellm/llms/azure/test_azure_common_utils.py - Merged upstream formatting + CARTO URL sanitization tests
  • ui/litellm-dashboard/src/app/layout.tsx - Merged ReactQueryProvider (upstream) + FeatureFlagsProvider (CARTO)

Synced (Required)

Files synced entirely from upstream:

  • .github/workflows/ghcr_helm_deploy.yml.txt - Deleted by upstream
  • .github/workflows/load_test.yml - Deleted by upstream
  • .github/workflows/llm-translation-testing.yml - Accepted upstream additions
  • tests/guardrails_tests/test_lakera_v2.py - New upstream tests, no CARTO changes

Fix Loop Interventions

Files synced due to repeated conflicts:

  • None

CARTO Features Verification

All patterns from .github/carto-features.yml verified:

  • OCI Gemini Tool Call UUIDs: uuid.uuid4().hex, _handle_tool_call_delta
  • Snowflake Streaming + Tool Calling: SnowflakeStreamingHandler, _transform_messages, tool_choice_value is None and tools
  • Snowflake Full URL Passthrough: CARTO: skip path construction if api_base already contains
  • Azure URL Suffix Stripping: chat/completions|completions|embeddings|audio/speech
  • JSON Repair for Streaming Tool Calls: _validate_and_repair_tool_arguments
  • Redis Session Storage: _store_session_in_redis, _patch_store_session_in_redis

Next Steps

  1. Review the CARTO customization decisions above
  2. Merge using "Create a merge commit"
  3. 🎉 Upstream sync complete!
🔧 Workflow Details (click to expand)

Workflow Run: https://github.com/CartoDB/litellm/actions/runs/24415740410

@Cartofante

Copy link
Copy Markdown
Collaborator Author

Caution

⚠️ Merge Method Reminder

Use "Create a merge commit" — Click the dropdown arrow next to merge button.

❌ Do NOT use "Squash and merge" or "Rebase and merge"

Squashing destroys upstream history and breaks future syncs (see PR #26).

@Cartofante

Copy link
Copy Markdown
Collaborator Author

📊 CARTO Feature Analysis Started

Mode: Analysis + Auto-fix if issues found

Analyzing how each CARTO customization was handled during conflict resolution.
This will explain WHY each decision was made.

Step Status
Extract CARTO PRs ⏳ In progress
Compare code versions ⏳ Pending
Analyze decisions ⏳ Pending
Generate report ⏳ Pending

View workflow →

@Cartofante

Copy link
Copy Markdown
Collaborator Author

✅ CARTO Feature Analysis Complete

Decision Count
Upstream Substitutes 0
Customized Upstream 1
Preserved CARTO 15
Incorrectly Dropped 0

Overall Assessment: PASS

📋 Full details in PR description above.


View workflow run → | Download analysis artifacts →

@mateo-di mateo-di self-assigned this Apr 20, 2026

@mateo-di mateo-di left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

QA done by @diezalina + approved by @anamanvil

@mateo-di
mateo-di merged commit febaa7e into carto/main Apr 20, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.