merge litellm_internal_staging - #25945
Merged
Sameerlite merged 404 commits intoApr 17, 2026
Merged
Conversation
…-standard-logging-object fix(azure/passthrough): populate standard_logging_object via logging hook
…ctions feat(mcp): expose per-server InitializeResult.instructions from gateway
…he-key fix(caching): add Responses API params to cache key allow-list
…-max-tokens feat(health-check): add BACKGROUND_HEALTH_CHECK_MAX_TOKENS env var
…nking-signature-retry feat(anthropic): retry /v1/messages after invalid thinking signature
…elds Boolean fields in the auto-generated guardrail provider form (e.g. Noma `use_v2`) rendered as empty Selects because the Form.Item only populated `initialValue` for percentage fields, and the `defaultValue` passed to the Select child was silently dropped by antd's controlled-component wrapper. Users could not tell what the backend default was, and the visual ambiguity made flags like `use_v2` look inoperative even though the save path worked. Unify `initialValue` to fall back through `fieldValue → field.default_value → (percentage ? 0.5 : undefined)`, and switch Select.Option values from "true"/"false" strings to real booleans so the backend default flows through without stringification.
…om-tool-schema feat(bedrock): normalize custom tool JSON schema for Invoke and Converse
…ution-pricing2 feat(gemini): Veo Lite pricing, video resolution usage and tiered cost
…_.py Cast message lists to the expected `List[Union[AllMessageValues, Message]]` type at `token_counter` call sites, and suppress the `no-redef` warning for the `compress` import in `__init__.py` caused by the wildcard `main` import. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
litellm_staging_04_04_2026
Litellm oss staging 04 11 2026
New MCPServer.instructions field requires a str; MagicMock attributes not explicitly set return a MagicMock object, which fails Pydantic validation.
…t-delete fix(ui): delete policy attachments via controlled modal
### Background The Gemini batchEmbedContents response handler hardcoded `index=0` for every embedding in the response. Any consumer relying on the OpenAI-format `index` field to match embeddings back to inputs would silently get wrong associations. ### Changes Use `enumerate` in `process_response` so each embedding gets its positional index instead of 0. ### Test Plan Added unit test asserting sequential indices and correct vector ordering for a 3-element batch response.
…hunk (#25533) * fix: emit input_json_delta for tool args bundled in first streaming chunk Some providers (xAI, Gemini) include tool_call function arguments in the same streaming chunk as the function name/id. The AnthropicStreamWrapper was discarding the trigger chunk entirely when starting a new content block, which silently dropped the input_json_delta carrying tool arguments. This caused tool_use blocks to arrive with empty input {}. Now queue the processed_chunk after content_block_start when it carries non-empty input_json_delta data. Backward compatible: providers that send empty arguments in the first chunk (OpenAI-style) are unaffected since the condition checks for truthy partial_json. * test: add tests for input_json_delta emission on bundled tool args Covers the fix for providers (xAI, Gemini) that bundle tool_call arguments in the same streaming chunk as the function name/id. Verifies the AnthropicStreamWrapper emits input_json_delta after content_block_start, and that empty-arg chunks (OpenAI-style) are unaffected. * style: apply Black formatting to streaming_iterator.py * fix: mirror input_json_delta fix to sync __next__ and add sync tests * test: make no_extra_delta tests assert explicitly instead of passing silently
…onfigured have no budget enforcement (#25557) * fix #25506 * address greptile review feedback * [Test] UI - Models: Add E2E tests for Add Model flow Add E2E tests covering: - Test connection with bad credentials shows failure modal - Adding a specific model and verifying it appears in All Models table - Adding a wildcard route and verifying it appears in All Models table - Verifying model dropdown shows provider-specific models (existing test updated) Added data-testid attributes to UI components to support stable test selectors. Tests verified passing 3/3 consecutive runs with zero flakiness. * address greptile review feedback (greploop iteration 1) Add cleanup helper to delete models created during tests, preventing stale data accumulation across repeated test runs. * fix CI: replace data-testid selectors with text/role-based selectors The data-testid attributes added to React components are not present in the CI-built UI output. Switch to using getByRole and getByText selectors which work with the rendered DOM regardless of build cache. * remove unnecessary cleanup helper The database is freshly seeded on every test run via seed.sql, so per-test cleanup is not needed. --------- Co-authored-by: Yuneng Jiang <yuneng@berri.ai> Co-authored-by: Krrish Dholakia <krrish+github@berri.ai>
* Serialize error message to a string; only scan last message * Update litellm/proxy/guardrails/guardrail_hooks/hiddenlayer/hiddenlayer.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * Add v2 of hiddenlayer guardrail implementation * Update litellm/proxy/guardrails/guardrail_hooks/hiddenlayer/hiddenlayer.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * Fix potential header issue * linting * Add image support --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
…t_latency strategy (#25548) * fix(router): discard oldest entry when trimming latency list in lowest_latency strategy The lowest_latency routing strategy keeps a rolling window of the most recent latency and time-to-first-token measurements per deployment. When the window is full, the strategy was discarding the *newest* value instead of the oldest, because the trim used `[: max_latency_list_size - 1]` (keeping indices 0..N-2) rather than `[1:]` (dropping index 0 and keeping indices 1..N-1). Since new values are appended at the end, the bug meant the most recent measurement was always dropped once the list reached capacity. The routing decisions then relied on stale data (including any early-spike values that never aged out), and timeout penalties written via `async_log_failure_event` were silently discarded as well. Fix the slice in all five call sites (sync + async log_success_event for both latency and time_to_first_token, and async_log_failure_event for the timeout penalty) and add regression tests covering each path. * test(router): cover async TTFT trim path in lowest_latency regression tests Adds test_ttft_list_trimming_discards_oldest_entry_async, an async counterpart to test_ttft_list_trimming_discards_oldest_entry that drives async_log_success_event with a ModelResponse and completion_start_time so the async time_to_first_token trim branch is actually exercised. Previously no test touched that code path: the sync TTFT test used log_success_event, and the async latency test passed a plain dict response_obj without stream/completion_start_time, so TTFT was never computed and the async trim was unreached. Verified load-bearing by reverting only the async TTFT slice — the new test fails and all others pass. * format
Litellm day 0 opus 4.7 support
[Infra] Merge dev branch
Litellm day 0 opus 4.7 support
Fix version in docs
[Infra] Bump llm_translation_testing resource class to xlarge
Litellm hotfix opus 4.7
…orker restarts Workers in llm_translation_testing have been crashing mid-run with "Not properly terminated" (OOM), even after bumping resource_class to xlarge. Reduce xdist workers from 8 to 4 to lower peak memory, and add --max-worker-restart=5 so a crashed worker is replaced instead of failing the whole run.
…ation_staging [Infra] Reduce llm_translation_testing parallelism and tolerate worker restarts
…ssertion Drop test_bedrock_invoke_messages_injects_thinking_for_clear_thinking_context_management. Its assertion 'interleaved-thinking-2025-05-14' in betas cannot hold because anthropic_beta_headers_config.json maps that header to null for the bedrock provider, so filter_and_transform_beta_headers drops it from the auto-added beta set before anthropic_beta is written to the request. The adjacent test_bedrock_invoke_messages_skips_thinking_injection_when_already_enabled already covers the inverse behavior for the same model, so no coverage is lost.
* Add announcement bar for Trivy compromise resolution notice Add a Docusaurus announcement bar to the top of the docs site informing users that the Trivy supply-chain compromise has been mitigated and resolved. The banner: - States all affected packages have been deleted and releases are safe - Links to the Security Townhall blog post for details - Links to the CI/CD v2 blog post for improvements made - Uses a green background with closeable dismiss button Co-authored-by: Krrish Dholakia <krrish-berri-2@users.noreply.github.com> * Use :::note admonition instead of announcement bar Replace the Docusaurus announcementBar with a :::note admonition on the docs index page. The note appears below the hero image with the title 'Security Update' and links to the Security Townhall and CI/CD v2 blog posts. Co-authored-by: Krrish Dholakia <krrish-berri-2@users.noreply.github.com> * Update security notice wording to 'contained' Co-authored-by: Krrish Dholakia <krrish-berri-2@users.noreply.github.com> * Move note above hero image and add to root page - Move the security notice above the product screenshot on /docs - Add the same notice to the root page (src/pages/index.md) Co-authored-by: Krrish Dholakia <krrish-berri-2@users.noreply.github.com> * Update security notice wording Co-authored-by: Krrish Dholakia <krrish-berri-2@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Krrish Dholakia <krrish-berri-2@users.noreply.github.com>
…gBetaTest [Test] Remove dead Bedrock clear_thinking interleaved-thinking-beta assertion
Three tests inherited by TestBedrockMoonshotInvoke from BaseLLMChatTest make live AWS Bedrock completion calls: test_developer_role_translation, test_message_with_name, and test_completion_cost. These have been crashing llm_translation_testing CI workers (reported as "failed on setup with worker 'gwN' crashed"). Replace each with a mocked override that intercepts the outgoing request via HTTPHandler.post / AsyncHTTPHandler.post patching: - test_developer_role_translation asserts the outgoing body maps the developer role to system (LiteLLM's translation for non-OpenAI providers). - test_message_with_name asserts the outgoing body preserves the user message. - test_completion_cost returns a canned moonshot-shaped response body with usage and asserts response_cost > 0 against the local model cost map. Follows the existing HTTPHandler + patch.object(client, "post") pattern used in test_bedrock_gpt_oss.py and test_bedrock_completion.py. No network traffic; the three tests now complete in ~0.3s.
…itellm_/amazing-almeida # Conflicts: # tests/test_litellm/llms/bedrock/messages/invoke_transformations/test_anthropic_claude3_transformation.py
TogetherAIConfig.get_supported_openai_params called get_model_info(), whose first line calls litellm.get_supported_openai_params() — which for together_ai routes straight back into this method. The recursion only terminated when Python's recursion limit was hit or when _get_model_info_helper raised "not mapped" at the deepest level. Either way the try/except caught it, so the bug stayed silent — but the cycle ran ~332 deep every time, emitting hundreds of DEBUG log lines per call. Surfaced as "infinite loop" in CI when the success_handler thread emitted that log spam against an already-closed stderr during test teardown. Replace the get_model_info() call with supports_function_calling(), which uses _get_model_info_helper directly and does not call get_supported_openai_params. Measured drop from 332 to 2 _get_model_info_helper calls per first uncached lookup. Also swap the test model from Qwen/Qwen3.5-9B (not in model_cost map) back to a mapped serverless model, Qwen/Qwen2.5-7B-Instruct-Turbo. The mapping gap is what made the recursion's tail end raise up into the success handler during teardown in the first place.
Extends the prior moonshot mocking to cover every inherited BaseLLMChatTest test that still made a live AWS Bedrock call. Adds request-body assertions for each override. New overrides: - test_content_list_handling: verifies the outgoing body round-trips user content in list-of-text form; asserts response.choices[0]. message.content parses back from the canned response. - test_pydantic_model_input: verifies a pydantic Message input does not raise and produces a parseable response. - test_response_format_type_text_with_tool_calls_no_tool_choice: verifies tools are forwarded and response_format + drop_params do not break the call. - test_streaming: verifies stream=True routes to the invoke-with-response-stream endpoint. Bedrock invoke streaming is intercepted at the make_sync_call import site rather than via the caller-supplied client, because CustomStreamWrapper.fetch_sync_stream invokes the stored make_call partial with client=litellm.module_level_client, overriding any client passed by the caller. Extracts a shared _make_moonshot_response helper and a _invoke_with_mocked_post harness so all the sync mocks share one canned response body. After this change TestBedrockMoonshotInvoke runs 23 passed, 29 skipped, 0 live-callers, all in under 1s locally.
[Test] Mock Bedrock Moonshot tests + [Fix] TogetherAIConfig recursion
bump: proxy extras version 0.4.65 → 0.4.66
bump: version 1.83.8 → 1.83.9
#25915) * Add capability to override default GitHub Copilot authentication endpoints This feature adds support for GitHub Enterprise subsriptions with custom domain/data ownership (which use a different URL compared to standard accounts) * Update documentation with new parameters * Move access token URL and Client ID retrieval outside for loop Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * Fix spurious comment from Greptile review * Align api_base retrieval behavior across chat and embedding transformations * Add missing GitHub Copilot client ID parameter in docs * Update website documentation with newer options for GitHub Enterprise Copilot * Fix default value for Copilot client ID in docs Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
CodeQL found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.
Contributor
|
Too many files changed for review. ( |
Sameerlite
merged commit Apr 17, 2026
27877b4
into
litellm_Sameerlite/openai-chat-to-responses
205 of 219 checks passed
|
|
fzowl
pushed a commit
to fzowl/litellm
that referenced
this pull request
Jun 24, 2026
merge litellm_internal_staging
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Relevant issues
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/test_litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewDelays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
CI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Screenshots / Proof of Fix
Type
🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test
Changes