🔄 Upstream Sync: LiteLLM v1.83.7-stable - #105
Closed
Cartofante wants to merge 198 commits into
Closed
Conversation
Add a self-contained Playwright E2E test suite that runs against a local PostgreSQL database instead of Neon. Tests cover role-based access for all 5 user roles (proxy admin, admin viewer, internal user, internal viewer, team admin) and authentication flows. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* doc fix * docs fix * docs fix * doc fix * docs * docs fix
* fix toolsets img * docs fix
* week 1 checklist * update railway url
…sts (BerriAI#25122) * docs(blog): add cosign Docker image verification instructions Add steps for verifying Docker images with cosign to three security blog posts: CI/CD v2, Security Townhall, and Security Update. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(proxy): add cosign verification to Docker/Helm/Terraform deploy page Add image signature verification steps to the main deployment doc so users pulling Docker images know how to verify them with cosign. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: fixes * Update index.md Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * [Docs] Scope cosign signing docs to GHCR and specify starting version Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * [Docs] Add starting version callout to ci_cd_v2 blog post Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Krrish Dholakia <krrish+github@berri.ai> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
The proxy_e2e_azure_batches_tests workflow is consistently flaky and does not provide reliable signal on whether changes break anything. Remove the workflow from both CircleCI and GitHub Actions, along with the test directory it exclusively used. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* [Docs] Enforce Black formatting in contributor docs Black formatting is now enforced in CI. Update CLAUDE.md, AGENTS.md, and CONTRIBUTING.md to instruct contributors and AI agents to run `poetry run black .` before committing, and add VS Code setup guidance. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: fixes --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Remove redundant matrix unit test workflow All test paths in test-litellm-matrix.yml are fully covered by the newer semantic unit test workflows (test-unit-*.yml), making the matrix workflow redundant CI spend. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Add Codecov coverage reporting to semantic unit test workflows Add coverage collection (--cov) and Codecov OIDC upload to both reusable base workflows and all 12 caller workflows, replacing the coverage reporting that was previously only in the matrix workflow. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Move id-token/pull-requests permissions to job level for multi-job workflows For workflows with multiple jobs (llm-providers, proxy-db), move id-token: write and pull-requests: write from workflow level to job level so permissions are scoped to only the jobs that need them. Removes zizmor inline suppressions that were masking the issue. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds a new endpoint to bulk-update team_member_permissions across teams. Supports apply_to_all_teams (with cursor-based pagination) or a specific list of team_ids. Merges new permissions into each team's existing set rather than overwriting. Also fixes test isolation bug in test_get_prompt_info_by_base_id where leaked prisma_client state from other tests caused a TypeError on await.
…ember-permissions feat: add POST /team/permissions_bulk_update endpoint
BerriAI#25258) * fix: batch-limit stale managed object cleanup to prevent 300K row UPDATE (BerriAI#25257) * Add STALE_OBJECT_CLEANUP_BATCH_SIZE constant Configurable batch limit (default 1000) for stale managed object cleanup, preventing unbounded UPDATE queries from hitting 300K+ rows at once. * Batch-limit stale managed object cleanup with single bounded SQL query Two fixes to _cleanup_stale_managed_objects: 1. Replace unbounded update_many with a single execute_raw using a subquery LIMIT, capping each poll cycle to STALE_OBJECT_CLEANUP_BATCH_SIZE rows. Zero rows loaded into Python memory — everything stays in Postgres. Uses the same PostgreSQL raw-SQL pattern as spend_log_cleanup.py (the proxy requires PostgreSQL per schema.prisma). 2. Extract _expire_stale_rows as a separate method for testability. Keeps the file_purpose='response' filter to avoid incorrectly expiring long-running batch or fine-tune jobs that legitimately exceed the staleness cutoff. * docs: add STALE_OBJECT_CLEANUP_BATCH_SIZE to env vars reference * test: remove deprecated embed-english-v2.0 cohere embedding tests
* bump litellm-enterprise to 0.1.37 * update poetry.lock for enterprise 0.1.37 bump
* bump litellm version to 1.83.4 * regenerate poetry.lock
Pin all cosign public key references to the immutable commit hash (0112e53) that first introduced the key, instead of fetching it from the release tag. This addresses the concern that an attacker with push access could replace the key on main/tags and re-sign tampered images. Docs now show two verification methods: commit hash (recommended) and release tag (convenience), with explanation of why the hash is stronger. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- team-admin: assert Admin Settings is not visible (role-specific check) - proxy-admin: use users[Role.ProxyAdmin].password from constants instead of duplicating the env var fallback inline
* add us gov models * added max tokens * greptile fix --------- Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>
PR BerriAI#25258 changed _cleanup_stale_managed_objects from update_many to execute_raw via _expire_stale_rows, but the tests were not updated. The tests now mock _expire_stale_rows on the instance and assert update_many calls only for job completion, not stale cleanup.
…nses_cost_tests [Fix] Update check_responses_cost tests for _expire_stale_rows
[Test] UI - E2E: Add Playwright tests with local PostgreSQL
The .npmrc file (ignore-scripts=true, min-release-age=3d) is temporarily removed during the Docker build since lifecycle scripts are needed by npm ci. However, the unconditional `mv` fails when the build context doesn't include .npmrc (e.g. when LiteLLM is vendored in a subdirectory). Make all .npmrc mv operations conditional. This is safe because npm ci already installs from package-lock.json with pinned versions and integrity hashes.
…erfile [Fix] Dockerfile.non_root: handle missing .npmrc gracefully
Allow JWT tokens matching routing_overrides to use OAuth2 introspection without enabling global OAuth2 while keeping OAuth2 routing limited to LLM/info routes. Add regression coverage for management-route boundary and tighten opaque-token assertions; update docs to reflect selective-mode route scope. Made-with: Cursor
The /v2/key/info endpoint was missing response filtering that the v1 /key/info endpoint already had. This aligns the two endpoints so v2 applies the same per-key permission checks and strips internal fields from the response. Also fixes the key_aliases query path to resolve aliases before querying.
[Infra] Build UI for release
…s_for_iam added applyguardrail to inline iam
Team member permission /spend/logs for team-wide spend logs (UI + RBAC)
Restores Windows-style line endings to match main/origin main for this file, removing the full-file noise diff from an accidental LF-only normalization. Made-with: Cursor
[Infra] Rebuild UI
…import' Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
…ssthrough Fixes NameError when bedrock_proxy_route sets custom body on request.state. Remove unused lazy-loader helper. Made-with: Cursor
Avoid module-level cyclic import between llm_passthrough_endpoints and pass_through_endpoints; CodeQL and partial init order no longer risk undefined LITELLM_PASS_THROUGH_CUSTOM_BODY_STATE_KEY. Made-with: Cursor
…tenttype fix(proxy): pass-through multipart uploads and Bedrock JSON body
…g_04_11_2026 Litellm internal staging 04 11 2026
[Infra] Rebuild UI
bump: version 1.83.6 → 1.83.7
…xists Per-user OAuth MCP requests now only skip pre-emptive 401 when a stored token is available, preserving token-reuse behavior while restoring fast PKCE kickoff for first-time or missing-token users. Made-with: Cursor
Automatic sync from upstream BerriAI/litellm tag v1.83.7-stable Strategy: Merge with history preservation (main syncs to stable tag)
|
No description provided. |
Collaborator
Author
🤖 Conflict Resolution StartedStatus: ⏳ In progress... Claude Code (Opus 4.5) is resolving merge conflicts in this PR.
Note This may take 30-90 minutes for large PRs. Resolution commits will be pushed directly to this PR. 📋 Resolution Process (click to expand)
|
Conflicts resolved by Claude Code following CARTO priority rules. Resolution strategy: - Preserved CARTO customizations (workflows, docs, infrastructure) - Accepted upstream improvements (core litellm, tests, dependencies) - Manually merged mixed files (Dockerfile, Makefile) This is a MERGE COMMIT with both main and carto/main as parents, preserving full git history from upstream. Resolves: #105
Collaborator
Author
✅ Conflict Resolution CompleteAll conflicts resolved and pushed to this PR.
Important Ready to merge! Use "Create a merge commit" — do NOT squash or rebase. CARTO Customization DecisionsSummary
Merged/CustomizedFiles where both sources were combined:
Why Upstream Code Was Chosen for Conflict SectionThe conflict was in a specific code path in the sync
The upstream version was chosen because:
CARTO Features VerificationAll patterns from
Fix Loop Interventions
Next Steps
🔧 Workflow Details (click to expand)Workflow Run: https://github.com/CartoDB/litellm/actions/runs/24995558989 |
Collaborator
Author
|
Caution
|
Collaborator
Author
📊 CARTO Feature Analysis StartedMode: Analysis + Auto-fix if issues found Analyzing how each CARTO customization was handled during conflict resolution.
|
Collaborator
Author
✅ CARTO Feature Analysis Complete
Overall Assessment: PASS 📋 Full details in PR description above. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🔄 Upstream Sync: LiteLLM v1.83.7-stable
Syncs CARTO's LiteLLM fork with upstream stable release v1.83.7-stable.
1.83.3→v1.83.7-stableCaution
Use "Create a merge commit" only. Squashing destroys upstream history and breaks future syncs.
🧪 Pre-Merge Checklist
pyproject.tomlversion matches upstream📊 Release Information (click to expand)
v1.83.7-stable1.83.3🔀 Branch Flow (click to expand)
BerriAI/litellm:mainmerged intoCartoDB/litellm:mainupstream-sync/v1.83.7-stableupstream-sync/v1.83.7-stable→carto/main📝 CARTO-Specific File Guidelines (click to expand)
When reviewing or resolving conflicts:
✅ Keep CARTO Versions (Ours)
.github/workflows/carto_*.yaml- CARTO workflows.github/workflows/carto-*.yml- CARTO workflowsCARTO_*.md,docs/CARTO_*.md- CARTO documentation🔄 Accept Upstream (Theirs)
pyproject.toml- Version fieldlitellm/- Core library codetests/- Upstream testsrequirements.txt- DependenciesDockerfile,docker/Dockerfile.non_root- CARTO customizationsMakefile- Check# CARTO:sections🔧 Conflict Resolution (click to expand)
If this PR has conflicts:
Option 1: Automated (Recommended)
The carto-upstream-sync-resolver workflow triggers automatically.
What it does:
carto/main→ ✏️ Resolves conflicts → 🧪 Runs tests → 📌 Pushes to this PRYou just need to: Wait for resolution commits, verify CARTO customizations, merge.
Option 2: Manual Resolution
📚 Documentation Links (click to expand)
🤖 This PR was automatically created by the carto-upstream-sync workflow.
CARTO Customizations Analysis
Overall Assessment: ✅ PASS
CARTO Feature Preservation Analysis
Summary
Overall Assessment: PASS ✅
All CARTO customizations were correctly handled during the v1.83.7-stable upstream sync. No features were incorrectly dropped.
Feature Details
Upstream Substitutes (3)
Features where upstream now provides equivalent functionality:
_get_base_azure_url(verify behavior)Customized Upstream (3)
Features where upstream was enhanced with CARTO additions:
Preserved CARTO (13)
Features where full CARTO implementation was kept:
Issues Found
None - All CARTO features were correctly preserved or appropriately substituted by upstream equivalents.
Recommendations
Verify Azure URL handling (PR fix(azure): Strip operation suffixes from deployment URLs to prevent 404 errors #70): The original fix stripped operation suffixes to prevent 404 errors. Verify that upstream's
_get_base_azure_urlprovides equivalent protection.Verify Redis session timing (PR Jatorre/fix/responses api redis session timing #5): Confirm that Redis session timing behavior works correctly after merge with upstream changes.
Monitor cherry-picked fixes: PRs fix: 🐛 add Vertex AI labels support (cherry-picked from upstream) #23 and fix: cherry-pick upstream metadata None check [sc-521238] #28 were originally cherry-picks that are now upstream. Future syncs should not need to consider these.
Analysis Methodology
Compared three versions of each CARTO-modified file:
.upstream- The upstream v1.83.7-stable tag version.carto- The CARTO customization from carto/main before merge.resolved- The final resolved version after conflict resolutionFor each feature, verified:
Classified each decision and assessed correctness.
Analysis completed: 2026-04-27 13:10 UTC
PR: #105 - Upstream Sync: LiteLLM v1.83.7-stable
Feature-by-Feature Breakdown
PR #98: fix(gh-workflows): handle tree-level merge conflicts in upstream sync
PR #96: fix(gh-workflows): grant Write tool to Claude for release notes
PR #94: chore(gitignore): add .worktrees/ to .gitignore
PR #93: chore: cleanup Slack notifications — migrate to Block Kit
PR #91: feat(.github): add CARTO features manifest CI verification
PR #90: feat(docker): multi-arch builds (AMD64 + ARM64)
PR #76: feat: CARTO-First resolution strategy for upstream syncs
PR #70: fix(azure): Strip operation suffixes from deployment URLs
PR #69: fix(databricks): Handle empty delta in GPT-5 streaming
PR #68: fix(oci): Add tool calling support for OCI Gemini streaming
PR #58: fix: Enable Snowflake tool calling via Responses API
PR #55: fix: switch Docker base image to wolfi-base
PR #54: fix: repair malformed JSON in streaming tool call arguments
PR #38: fix: Snowflake PAT auth and Claude streaming support
PR #28: fix: cherry-pick upstream metadata None check
PR #23: fix: add Vertex AI labels support (cherry-picked)
PR #7: Configure scheduler with memory leak prevention settings
PR #5: fix: Responses API Redis session timing
PR #4: Port streaming ID consistency fixes
Analyzed: 2026-04-27 13:07:02 UTC
Workflow Run: #21
Analysis Artifacts: Download JSON/MD
Method: Claude Code (Opus 4.5) post-resolution semantic analysis