[Infra] Promote Internal Staging to main - #27436
Merged
Merged
Conversation
Container ownership and skill rows are looked up on every retrieve / delete / list / file-content / chat-completion-with-skill call. The new stores wrapped raw Prisma queries with no cache, putting one DB round-trip on each request. Add an in-process TTL'd cache mirroring the _byok_cred_cache pattern in mcp_server/server.py: per-key (value, monotonic_timestamp), 60s TTL, 10000-entry cap with full-clear on overflow, invalidated by every write. Negative results (`None`) are cached too so untracked-resource checks also skip the DB. Tests cover: cache-after-first-hit, negative caching, write invalidation, no-caching-on-DB-error, TTL expiry, capacity eviction. 56 tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…itellm_fix_scim_virtual_key_deactivation
…itellm_suppress-spend-log-tracebacks-2208
A SCIM PUT may legally omit `active` (full-replace with the field absent). Pydantic fills the SCIMUser.active default of True, so the PUT handler was overwriting metadata.scim_active with True even when the client never sent it — silently reactivating a previously SCIM-blocked user and unblocking their keys. Use model_fields_set to detect whether the client actually sent `active`. If omitted, preserve the prior scim_active value and skip the cascade to virtual keys. Also drop comments added in this PR that just narrate what the code does; keep only the docstrings and the SQL-NULL pitfall note that explain non-obvious behaviour. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
fix(router): constrain same-name deployment routing by access groups
…t_thinking fix(gemini): follow provider defaults for Gemini 3 thinking
…itellm_hotfix_gpt-5.5-minimal-flag # Conflicts: # tests/test_litellm/llms/vertex_ai/test_vertex_ai_common_utils.py Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
feat(mcp): enforce org-level MCP server and toolset permissions
Newer Cloudflare Workers AI models (e.g. Nemotron) emit 'response_text' instead of 'response' on streamed chunks. The non-streaming path was already updated to fall back to 'response_text' (#26385), but the streaming chunk parser still only read 'response', which caused streaming requests against those models to silently produce empty content. Mirror the non-streaming fallback in CloudflareChatResponseIterator.chunk_parser and add a streaming test for the response_text shape. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
…itellm_oss_staging_04_25_2026 Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
…eam-passthrough fix(guardrails): preserve responses event streams in presidio output masking
chore(staging): roll oss_staging_04_25_2026 into internal staging (output_config fix + 4 upstream sync fixes)
…m_id - Remove unused _encode_gcp_label_value / _decode_gcp_label_value singular helpers; only the _chunks variants are actually called. - Use 'is not None' check for custom_id so empty-string custom_ids are still labeled and round-trip through batch outputs. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
…itellm_vertex-batch-output-transformation Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Co-authored-by: Sameer Kankute <Sameerlite@users.noreply.github.com>
…_output_to_openai The method was overwriting logging_obj.optional_params, logging_obj.model, and logging_obj.start_time on the caller's Logging instance. When invoked from llm_http_handler.py's generic framework path, the framework's own logging_obj (which already went through pre_call) had its properties clobbered, causing model and start_time to reflect the last batch line's values rather than the original call context. Fix: create a fresh local Logging instance for the per-line transformation instead of mutating the incoming logging_obj. The caller's object is now left entirely untouched regardless of whether a logging_obj was passed in or not. Regression tests added to verify model, start_time, and optional_params are not mutated on the caller's logging_obj. Co-authored-by: Sameer Kankute <Sameerlite@users.noreply.github.com>
Adds litellm.disable_vertex_batch_output_transformation (default False). When True, afile_content returns raw Vertex predictions.jsonl untouched so users that parse candidates/modelVersion directly are not broken.
…fort="none"
Setting reasoning_effort="none" on Anthropic chat models (direct, Bedrock
Invoke, Bedrock Converse, Vertex AI Anthropic, Azure AI Anthropic) crashed
LiteLLM with:
litellm.APIConnectionError: 'NoneType' object has no attribute 'get'
Both the Anthropic chat transformation and Bedrock Converse called
``AnthropicConfig._map_reasoning_effort`` and assigned the ``None`` it returns
for ``"none"`` directly to ``optional_params["thinking"]``. Downstream
``is_thinking_enabled`` then did ``optional_params["thinking"].get("type")``
and crashed.
Pop ``thinking`` (and on Claude 4.6/4.7, ``output_config``) instead of
assigning ``None``, restoring the documented contract that
``reasoning_effort="none"`` means "do not enable thinking". This also
prevents downstream Anthropic 400s ("thinking: Input should be an object",
"output_config.effort: Input should be ...") if the bug were ever masked.
Verified end-to-end against the live Anthropic API and Bedrock Converse
on claude-opus-4-{5,6,7} and claude-sonnet-4-6, plus Bedrock Invoke for
Claude 4.5/4.6. Vertex AI Anthropic and Azure AI Anthropic inherit the
fixed ``map_openai_params`` from ``AnthropicConfig`` and need no further
changes.
…ansformation feat(vertex-ai): transform batch prediction outputs to OpenAI format
…itellm_/nifty-kilby-82870d # Conflicts: # uv.lock
[Infra] Packaging: Relax Core Runtime Pins To Ranges
Pin the new digest published 2026-05-05: sha256:31da6565... (from sha256:3258be47...). Delta in baseline packages: zlib 1.3.2-r2 -> 1.3.2-r3. glibc stays at 2.43-r7 (still the latest available; whatever further glibc fixes for CVE-2026-5450 / CVE-2026-5928 land in -r8+ from Chainguard, this PR doesn't touch those). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Empirical grype scan of the built runtime image flagged ip-address@10.1.0 (Medium) bundled inside /usr/local/lib/node_modules/npm. npm@11.14.0 bundles ip-address@10.1.1 which carries the fix. Verified by rebuilding the image and rescanning: ip-address finding gone. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Revert disk_cache.py JSONDisk swap + remove test_disk_cache.py. The JSONDisk migration is backwards-incompatible (existing pickle caches become unreadable; non-JSON values raise unguarded TypeError on set) and warrants its own focused PR with a feature flag rather than riding along with the CVE/dep-bump sweep. CVE-2025-69872 remains unmitigated at the diskcache layer; users concerned about pickle-RCE on cache-dir writers can avoid Cache(type="disk") or pin a fork until upstream ships a fix. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The entry was added to cover the now-reverted black 24.10.0 -> 26.3.1 bump. With the bump dropped, upstream's existing liccheck setup is unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…puts->steps Google updated their Interactions OpenAPI spec (https://ai.google.dev/static/api/interactions.openapi.json), removing the readOnly 'outputs' property from CreateModelInteractionParams in favor of 'steps' (a polymorphic transcript array). The compliance test fetches the live spec, so it began failing on every PR once the spec flipped over. Update the asserted output-field list to match. Note: this only re-aligns the spec-shape assertion. Our SDK response types (litellm/types/interactions/generated.py) still expose 'outputs' and need to be regenerated separately to add 'steps'/Step variants and decide on a back-compat path for callers reading .outputs.
…alth-app fix: remove separate health app
test(interactions): align openapi compliance with upstream rename outputs->steps
[Security] Clear AWS Inspector CVE findings on Docker image
Merge of cve-sweep-2026-05 into litellm_yj_may7 picked the older npm@11.12.1 line, regressing the bump in f08b1b6 that cleared ip-address GHSA-v2v4-37r5-5v8g (npm@11.12.1 bundles ip-address@10.1.0; 11.14.0 bundles 10.1.1).
[Infra] Merge dev branch
[Infra] Bump versions
Contributor
|
Too many files changed for review. ( |
|
|
…aging Restore lineage between main and internal staging so the next staging->main promotion (#27436) can merge without conflicts. main was 2 commits ahead: - 6ff668c squash-merge of the previous staging->main promotion (#27245) - 8c9830e feat(xai): add grok-4.3 (#27396), already present on staging The squash-merge has no shared lineage with the individual commits that went into staging, which is why git surfaced 13 textual conflicts despite both sides having the same logical content. Every conflicting file's main-side change came from 6ff668c only, and the matching staging-side changes are the post-promotion evolution. Resolved all 13 with --ours (staging's version is the latest evolution; main's snapshot is stale). The grok-4.3 entries auto-merged in model_prices_and_context_window.json and its backup, but were already on staging via an independent commit, so the net diff vs HEAD is empty for those files. Net new content from this merge: 12 lines added to ui/litellm-dashboard/package-lock.json -- npm 11 libc array tags on four existing entries, no functional impact.
2 tasks
chore: merge main into internal_staging to restore lineage
Contributor
ishaan-berri
approved these changes
May 8, 2026
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
yuneng-berri
enabled auto-merge
May 8, 2026 01:05
shin-berri
approved these changes
May 8, 2026
fzowl
pushed a commit
to fzowl/litellm
that referenced
this pull request
Jun 24, 2026
…aging Restore lineage between main and internal staging so the next staging->main promotion (BerriAI#27436) can merge without conflicts. main was 2 commits ahead: - 6ff668c squash-merge of the previous staging->main promotion (BerriAI#27245) - 8c9830e feat(xai): add grok-4.3 (BerriAI#27396), already present on staging The squash-merge has no shared lineage with the individual commits that went into staging, which is why git surfaced 13 textual conflicts despite both sides having the same logical content. Every conflicting file's main-side change came from 6ff668c only, and the matching staging-side changes are the post-promotion evolution. Resolved all 13 with --ours (staging's version is the latest evolution; main's snapshot is stale). The grok-4.3 entries auto-merged in model_prices_and_context_window.json and its backup, but were already on staging via an independent commit, so the net diff vs HEAD is empty for those files. Net new content from this merge: 12 lines added to ui/litellm-dashboard/package-lock.json -- npm 11 libc array tags on four existing entries, no functional impact.
fzowl
pushed a commit
to fzowl/litellm
that referenced
this pull request
Jun 24, 2026
[Infra] Promote Internal Staging to main
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Relevant issues
Linear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/test_litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewDelays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
CI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Screenshots / Proof of Fix
Type
🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test
Changes