Skip to content

feat(a2a): watsonx Orchestrate agent provider - #29410

Merged
Sameerlite merged 14 commits into
litellm_internal_stagingfrom
litellm_watsonx_orchestrate_agents
Jun 2, 2026
Merged

feat(a2a): watsonx Orchestrate agent provider#29410
Sameerlite merged 14 commits into
litellm_internal_stagingfrom
litellm_watsonx_orchestrate_agents

Conversation

@Sameerlite

@Sameerlite Sameerlite commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add watsonx_orchestrate A2A provider (auth, submit run, poll, SSE + fallback streaming).
  • Register provider in A2AProviderConfigManager.
  • Add dashboard agent type in agent_create_fields.json.
  • Unit tests for WXO request/response transformation helpers.
    Fix LIT-1967

Test plan

  • pytest tests/test_litellm/a2a_protocol/providers/watsonx_orchestrate/test_watsonx_orchestrate_transformation.py -v
  • Create agent in UI with watsonx Orchestrate type and invoke via /a2a/{agent}/message/send
image image

Note

Medium Risk
Introduces new outbound auth (API keys, bearer tokens) and long-running poll/stream HTTP to customer CP4D/IBM endpoints; failures are mostly isolated to the new provider but misconfiguration can leak credentials into logs if existing patterns do.

Overview
Adds a watsonx_orchestrate A2A provider so LiteLLM can drive IBM watsonx Orchestrate agents through the existing agent/A2A path.

Provider wiring: A2AProviderConfigManager resolves custom_llm_provider=watsonx_orchestrate to a new config/handler stack that translates A2A send/stream requests into WXO REST calls (POST /v1/orchestrate/runs, poll GET .../runs/{id}, optional POST .../runs/stream), then maps results back to A2A JSON-RPC (non-streaming message responses and streaming task/status/artifact events, including fake streaming when the upstream response is already complete).

Auth & params: Supports CP4D (username + API key → cluster authorize) and IBM Cloud IAM (API key only), with in-memory bearer token caching. Required litellm_params: cp4d_host, instance_id, wxo_agent_id, api_key; optional auth_mode, thread_id, username (enforced at runtime for CP4D).

Proxy UI: agent_create_fields.json gains a watsonx Orchestrate agent type with credential fields and custom_llm_provider: watsonx_orchestrate.

Tests: New unit tests cover transformations, token TTL, SSE accumulation, streaming poll/failure paths, config manager registration, and dashboard field defaults.

Reviewed by Cursor Bugbot for commit 900926a. Bugbot is set up for automated code reviews on this repo. Configure here.

Bridge A2A message/send to WXO runs API (CP4D and IBM Cloud IAM auth),
with dashboard agent type metadata and unit tests for transformations.

Co-authored-by: Cursor <cursoragent@cursor.com>
@greptile-apps

greptile-apps Bot commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds a watsonx_orchestrate A2A provider that bridges LiteLLM's A2A path to IBM watsonx Orchestrate's REST runs API, including CP4D and IBM Cloud IAM auth with in-memory token caching, poll/SSE streaming, and a fake-streaming fallback.

  • New provider module (handler.py, config.py, transformation.py) wires WXO auth, run submission, polling, and SSE accumulation through litellm's existing get_async_httpx_client infrastructure; the A2AProviderConfigManager registers the provider with a lazy import.
  • The dashboard agent_create_fields.json gains a watsonx_orchestrate entry with credential fields matching the handler's required litellm_params.
  • Unit tests cover transformations, token TTL/caching, SSE accumulation, poll-timeout, streaming fallback, and the config manager registration — all mocked, no real network calls.

Confidence Score: 5/5

The PR is safe to merge; the new provider is self-contained, uses litellm's shared HTTP infrastructure, and does not touch any existing request paths.

All three major issues flagged in the previous review round have been corrected: the streaming path now uses get_async_httpx_client instead of a raw httpx.AsyncClient(verify=False), the try/except httpx.TransportError block covers only the initial POST, and the CP4D expiration field is now correctly treated as an absolute Unix timestamp. The remaining findings are style-level redundancies that do not affect correctness.

No files require special attention; handler.py carries the most complexity but the logic is well-tested and the auth paths are correct.

Important Files Changed

Filename Overview
litellm/a2a_protocol/providers/watsonx_orchestrate/handler.py Core handler with auth, polling, SSE accumulation, and streaming fallback; previously flagged issues have been addressed; explicit response.raise_for_status() after client.post() is redundant since AsyncHTTPHandler.post already calls it internally
litellm/a2a_protocol/providers/watsonx_orchestrate/transformation.py Transformation helpers for A2A↔WXO mapping and fake streaming; well-structured with multiple fallback paths for text extraction
litellm/a2a_protocol/providers/config_manager.py Registers watsonx_orchestrate provider with lazy import; follows the same pattern as BedrockAgentCore
litellm/proxy/public_endpoints/agent_create_fields.json Adds watsonx Orchestrate dashboard agent type; field definitions match the handler's _extract_litellm_params requirements
tests/test_litellm/a2a_protocol/providers/watsonx_orchestrate/test_watsonx_orchestrate_transformation.py Good mock-only test coverage for transformation helpers, token TTL, cache behavior, SSE accumulation, and streaming fallback/poll paths; assertions are correct

Reviews (11): Last reviewed commit: "fix(a2a/wxo): evict expired token cache ..." | Re-trigger Greptile

Comment thread litellm/a2a_protocol/providers/watsonx_orchestrate/handler.py Outdated
@codecov

codecov Bot commented Jun 1, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.84758% with 30 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
..._protocol/providers/watsonx_orchestrate/handler.py 91.66% 14 Missing ⚠️
...a_protocol/providers/watsonx_orchestrate/config.py 40.00% 9 Missing ⚠️
...ol/providers/watsonx_orchestrate/transformation.py 91.56% 7 Missing ⚠️

📢 Thoughts on this report? Let us know!

Comment thread litellm/a2a_protocol/providers/watsonx_orchestrate/handler.py Outdated
@veria-ai

veria-ai Bot commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

Route WXO streaming through get_async_httpx_client (TLS verification
enabled). Cache bearer tokens with TTL buffer. Extract A2A reply text via
a dedicated helper instead of hard-coded JSON paths.

Co-authored-by: Cursor <cursoragent@cursor.com>
@Sameerlite

Copy link
Copy Markdown
Contributor Author

@greptileai

Comment thread litellm/a2a_protocol/providers/watsonx_orchestrate/handler.py
CP4D /authorize returns expiration as epoch seconds, not TTL. Compute
remaining lifetime against wall clock so cached tokens refresh before expiry.

Co-authored-by: Cursor <cursoragent@cursor.com>
@Sameerlite

Copy link
Copy Markdown
Contributor Author

@greptileai Addressed P1: CP4D expiration is now interpreted as an absolute Unix timestamp (expiration - time.time()), not added to time.monotonic(). IBM Cloud IAM expires_in path unchanged. Added unit test test_cp4d_token_ttl_from_absolute_expiration.

Co-authored-by: Cursor <cursoragent@cursor.com>
@Sameerlite

Copy link
Copy Markdown
Contributor Author

@greptileai

@Sameerlite

Copy link
Copy Markdown
Contributor Author

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 3 potential issues.

Autofix Details

Bugbot Autofix prepared fixes for all 3 issues found in the latest run.

  • ✅ Fixed: Text extraction elif matches non-text kind parts
    • A2A text extraction now only includes parts whose kind is text, missing, or empty, with regression coverage for non-text parts carrying text fields.
  • ✅ Fixed: SSE parser crashes on non-dict JSON events
    • WXO result text extraction now returns an empty string for non-dict JSON values, so SSE accumulation ignores keepalive-style events without aborting.
  • ✅ Fixed: Token cache minimum can serve expired tokens
    • Token cache expiry now floors at zero after the safety buffer, preventing short-lived tokens from being served past their actual expiration.

You can send follow-ups to the cloud agent here.

Comment thread litellm/a2a_protocol/providers/watsonx_orchestrate/handler.py Outdated
@CLAassistant

CLAassistant commented Jun 1, 2026

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
2 out of 3 committers have signed the CLA.

✅ Sameerlite
✅ mateo-berri
❌ cursoragent
You have signed the CLA already but the status is still pending? Let us recheck it.

@Sameerlite

Copy link
Copy Markdown
Contributor Author

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

  • ✅ Fixed: Broad exception catch silently swallows streaming errors causing double requests
    • Restricted the WXO streaming fallback to transport-level failures so HTTP/status, JSON parsing, and other unexpected errors propagate without a second request.
Preview (3fcbc4ceac)
diff --git a/litellm/a2a_protocol/providers/config_manager.py b/litellm/a2a_protocol/providers/config_manager.py
--- a/litellm/a2a_protocol/providers/config_manager.py
+++ b/litellm/a2a_protocol/providers/config_manager.py
@@ -48,4 +48,11 @@
 
             return BedrockAgentCoreA2AConfig()
 
+        if custom_llm_provider == "watsonx_orchestrate":
+            from litellm.a2a_protocol.providers.watsonx_orchestrate.config import (
+                WatsonxOrchestrateA2AConfig,
+            )
+
+            return WatsonxOrchestrateA2AConfig()
+
         return None

diff --git a/litellm/a2a_protocol/providers/watsonx_orchestrate/__init__.py b/litellm/a2a_protocol/providers/watsonx_orchestrate/__init__.py
new file mode 100644
--- /dev/null
+++ b/litellm/a2a_protocol/providers/watsonx_orchestrate/__init__.py
@@ -1,0 +1,3 @@
+"""
+IBM watsonx Orchestrate (WXO) A2A provider.
+"""

diff --git a/litellm/a2a_protocol/providers/watsonx_orchestrate/config.py b/litellm/a2a_protocol/providers/watsonx_orchestrate/config.py
new file mode 100644
--- /dev/null
+++ b/litellm/a2a_protocol/providers/watsonx_orchestrate/config.py
@@ -1,0 +1,55 @@
+"""
+A2A provider configuration for IBM watsonx Orchestrate (WXO).
+"""
+
+from typing import Any, AsyncIterator, Dict, Optional
+
+from litellm.a2a_protocol.providers.base import BaseA2AProviderConfig
+from litellm.a2a_protocol.providers.watsonx_orchestrate.handler import (
+    WatsonxOrchestrateHandler,
+)
+
+
+class WatsonxOrchestrateA2AConfig(BaseA2AProviderConfig):
+    """A2A bridge for IBM watsonx Orchestrate (REST runs API + poll/SSE)."""
+
+    async def handle_non_streaming(
+        self,
+        request_id: str,
+        params: Dict[str, Any],
+        api_base: Optional[str] = None,
+        **kwargs: Any,
+    ) -> Dict[str, Any]:
+        """Handle a non-streaming A2A request via WXO runs API."""
+        litellm_params = kwargs.get("litellm_params")
+        if not litellm_params:
+            raise ValueError(
+                "litellm_params is required for WatsonxOrchestrateA2AConfig "
+                "(must contain cp4d_host, instance_id, wxo_agent_id, api_key)"
+            )
+        return await WatsonxOrchestrateHandler.handle_non_streaming(
+            request_id=request_id,
+            params=params,
+            litellm_params=litellm_params,
+        )
+
+    async def handle_streaming(
+        self,
+        request_id: str,
+        params: Dict[str, Any],
+        api_base: Optional[str] = None,
+        **kwargs: Any,
+    ) -> AsyncIterator[Dict[str, Any]]:
+        """Handle a streaming A2A request via WXO streaming runs API."""
+        litellm_params = kwargs.get("litellm_params")
+        if not litellm_params:
+            raise ValueError(
+                "litellm_params is required for WatsonxOrchestrateA2AConfig "
+                "(must contain cp4d_host, instance_id, wxo_agent_id, api_key)"
+            )
+        async for chunk in WatsonxOrchestrateHandler.handle_streaming(
+            request_id=request_id,
+            params=params,
+            litellm_params=litellm_params,
+        ):
+            yield chunk

diff --git a/litellm/a2a_protocol/providers/watsonx_orchestrate/handler.py b/litellm/a2a_protocol/providers/watsonx_orchestrate/handler.py
new file mode 100644
--- /dev/null
+++ b/litellm/a2a_protocol/providers/watsonx_orchestrate/handler.py
@@ -1,0 +1,360 @@
+"""
+Handler for IBM watsonx Orchestrate (WXO) agent provider.
+"""
+
+import asyncio
+import hashlib
+import json
+import time
+from typing import Any, AsyncIterator, Dict, Optional, Tuple, cast
+
+import httpx
+
+from litellm._logging import verbose_logger
+from litellm.a2a_protocol.providers.watsonx_orchestrate.transformation import (
+    WatsonxOrchestrateTransformation,
+)
+from litellm.llms.custom_httpx.http_handler import (
+    AsyncHTTPHandler,
+    get_async_httpx_client,
+)
+from litellm.types.llms.custom_http import httpxSpecialProvider
+
+_IBM_CLOUD_IAM_URL = "https://iam.cloud.ibm.com/identity/token"
+_POLL_INTERVAL_S = 2.0
+_MAX_POLL_ATTEMPTS = 90
+_TOKEN_CACHE_TTL_BUFFER_S = 60
+_token_cache: Dict[str, Tuple[str, float]] = {}
+
+
+class WatsonxOrchestrateHandler:
+    @staticmethod
+    def _http_client(timeout: float = 90.0) -> AsyncHTTPHandler:
+        return get_async_httpx_client(
+            llm_provider=cast(Any, httpxSpecialProvider.A2AProvider),
+            params={"timeout": timeout},
+        )
+
+    @staticmethod
+    def _token_cache_key(
+        auth_mode: str,
+        cp4d_host: str,
+        api_key: str,
+        username: Optional[str],
+    ) -> str:
+        material = f"{auth_mode}:{cp4d_host}:{username or ''}:{api_key}"
+        return hashlib.sha256(material.encode()).hexdigest()
+
+    @staticmethod
+    def _cp4d_token_ttl_seconds(
+        expiration: Any, now_wall: Optional[float] = None
+    ) -> int:
+        # CP4D returns expiration as absolute Unix epoch seconds, not a duration.
+        expires_at = int(expiration)
+        wall = now_wall if now_wall is not None else time.time()
+        return max(expires_at - int(wall), 0)
+
+    @staticmethod
+    async def _get_bearer_token(
+        cp4d_host: str,
+        auth_mode: str,
+        api_key: str,
+        username: Optional[str] = None,
+        client: Optional[AsyncHTTPHandler] = None,
+    ) -> str:
+        cache_key = WatsonxOrchestrateHandler._token_cache_key(
+            auth_mode, cp4d_host, api_key, username
+        )
+        now = time.monotonic()
+        cached = _token_cache.get(cache_key)
+        if cached and cached[1] > now:
+            return cached[0]
+
+        if client is None:
+            client = WatsonxOrchestrateHandler._http_client(timeout=30.0)
+
+        if auth_mode == "ibm_cloud":
+            response = await client.post(
+                _IBM_CLOUD_IAM_URL,
+                data={
+                    "grant_type": "urn:ibm:params:oauth:grant-type:apikey",
+                    "apikey": api_key,
+                },
+                headers={"Content-Type": "application/x-www-form-urlencoded"},
+            )
+            response.raise_for_status()
+            payload = response.json()
+            token = str(payload["access_token"])
+            ttl_s = int(payload.get("expires_in", 3600))
+        else:
+            if not username:
+                raise ValueError(
+                    "'username' is required in litellm_params when auth_mode='cp4d'"
+                )
+            token_url = f"{cp4d_host.rstrip('/')}/icp4d-api/v1/authorize"
+            response = await client.post(
+                token_url,
+                json={"username": username, "api_key": api_key},
+                headers={"Content-Type": "application/json"},
+            )
+            response.raise_for_status()
+            payload = response.json()
+            token = str(payload["token"])
+            expiration = payload.get("expiration")
+            if expiration is None:
+                ttl_s = 3600
+            else:
+                ttl_s = WatsonxOrchestrateHandler._cp4d_token_ttl_seconds(expiration)
+
+        expires_at = now + max(ttl_s - _TOKEN_CACHE_TTL_BUFFER_S, 0)
+        _token_cache[cache_key] = (token, expires_at)
+        return token
+
+    @staticmethod
+    async def _poll_run(
+        base_url: str,
+        run_id: str,
+        auth_headers: Dict[str, str],
+        client: AsyncHTTPHandler,
+        max_attempts: int = _MAX_POLL_ATTEMPTS,
+        interval_s: float = _POLL_INTERVAL_S,
+    ) -> Dict[str, Any]:
+        url = f"{base_url}/v1/orchestrate/runs/{run_id}"
+
+        for attempt in range(max_attempts):
+            await asyncio.sleep(interval_s)
+            response = await client.get(url, headers=auth_headers)
+            response.raise_for_status()
+            result: Dict[str, Any] = response.json()
+            status = result.get("status", "")
+            verbose_logger.debug(
+                f"WXO: Poll {attempt + 1}/{max_attempts} run='{run_id}' status='{status}'"
+            )
+            if status in WatsonxOrchestrateTransformation.TERMINAL_STATES:
+                return result
+
+        raise TimeoutError(
+            f"WXO run '{run_id}' did not reach a terminal state after "
+            f"{max_attempts * interval_s:.0f}s"
+        )
+
+    @staticmethod
+    async def _accumulate_wxo_sse_text(response: Any) -> str:
+        accumulated_text = ""
+        async for line in response.aiter_lines():
+            if not line.startswith("data:"):
+                continue
+            data_str = line[5:].strip()
+            if not data_str or data_str == "[DONE]":
+                continue
+            try:
+                event = json.loads(data_str)
+            except json.JSONDecodeError:
+                continue
+            chunk_text = WatsonxOrchestrateTransformation.extract_text_from_wxo_result(
+                event
+            )
+            if chunk_text:
+                accumulated_text += chunk_text
+        return accumulated_text
+
+    @staticmethod
+    def _extract_litellm_params(litellm_params: Dict[str, Any]) -> tuple:
+        cp4d_host = litellm_params.get("cp4d_host") or ""
+        instance_id = litellm_params.get("instance_id") or ""
+        wxo_agent_id = litellm_params.get("wxo_agent_id") or ""
+        api_key = litellm_params.get("api_key") or ""
+        username = litellm_params.get("username") or None
+        auth_mode = litellm_params.get("auth_mode") or "cp4d"
+        thread_id = litellm_params.get("thread_id") or None
+
+        if not cp4d_host:
+            raise ValueError("'cp4d_host' is required in litellm_params for WXO agents")
+        if not instance_id:
+            raise ValueError(
+                "'instance_id' is required in litellm_params for WXO agents"
+            )
+        if not wxo_agent_id:
+            raise ValueError(
+                "'wxo_agent_id' is required in litellm_params for WXO agents"
+            )
+        if not api_key:
+            raise ValueError("'api_key' is required in litellm_params for WXO agents")
+
+        return (
+            cp4d_host,
+            instance_id,
+            wxo_agent_id,
+            api_key,
+            username,
+            auth_mode,
+            thread_id,
+        )
+
+    @staticmethod
+    async def handle_non_streaming(
+        request_id: str,
+        params: Dict[str, Any],
+        litellm_params: Dict[str, Any],
+    ) -> Dict[str, Any]:
+        (
+            cp4d_host,
+            instance_id,
+            wxo_agent_id,
+            api_key,
+            username,
+            auth_mode,
+            thread_id,
+        ) = WatsonxOrchestrateHandler._extract_litellm_params(litellm_params)
+
+        client = WatsonxOrchestrateHandler._http_client(timeout=90.0)
+        token = await WatsonxOrchestrateHandler._get_bearer_token(
+            cp4d_host=cp4d_host,
+            auth_mode=auth_mode,
+            api_key=api_key,
+            username=username,
+            client=client,
+        )
+        base_url = WatsonxOrchestrateTransformation.get_api_base_url(
+            cp4d_host, instance_id
+        )
+        auth_headers = {
+            "Authorization": f"Bearer {token}",
+            "Content-Type": "application/json",
+            "Accept": "application/json",
+        }
+
+        text = WatsonxOrchestrateTransformation.extract_text_from_a2a_params(params)
+        body = WatsonxOrchestrateTransformation.build_wxo_run_body(
+            wxo_agent_id=wxo_agent_id, text=text, thread_id=thread_id
+        )
+
+        run_response = await client.post(
+            f"{base_url}/v1/orchestrate/runs",
+            json=body,
+            headers=auth_headers,
+        )
+        run_response.raise_for_status()
+        run_data: Dict[str, Any] = run_response.json()
+
+        status = run_data.get("status", "")
+        if status not in WatsonxOrchestrateTransformation.TERMINAL_STATES:
+            run_id = run_data.get("run_id") or run_data.get("id") or ""
+            if not run_id:
+                raise ValueError(f"WXO: No run_id in response: {run_data}")
+            run_data = await WatsonxOrchestrateHandler._poll_run(
+                base_url=base_url,
+                run_id=run_id,
+                auth_headers=auth_headers,
+                client=client,
+            )
+            status = run_data.get("status", "")
+
+        if status not in WatsonxOrchestrateTransformation.SUCCESS_STATES:
+            raise RuntimeError(
+                f"WXO run ended with non-success status '{status}': {run_data}"
+            )
+
+        response_text = WatsonxOrchestrateTransformation.extract_text_from_wxo_result(
+            run_data
+        )
+        return WatsonxOrchestrateTransformation.build_a2a_message_response(
+            request_id=request_id, text=response_text
+        )
+
+    @staticmethod
+    async def handle_streaming(
+        request_id: str,
+        params: Dict[str, Any],
+        litellm_params: Dict[str, Any],
+        chunk_size: int = 50,
+        delay_ms: int = 10,
+    ) -> AsyncIterator[Dict[str, Any]]:
+        (
+            cp4d_host,
+            instance_id,
+            wxo_agent_id,
+            api_key,
+            username,
+            auth_mode,
+            thread_id,
+        ) = WatsonxOrchestrateHandler._extract_litellm_params(litellm_params)
+
+        client = WatsonxOrchestrateHandler._http_client(timeout=120.0)
+        token = await WatsonxOrchestrateHandler._get_bearer_token(
+            cp4d_host=cp4d_host,
+            auth_mode=auth_mode,
+            api_key=api_key,
+            username=username,
+            client=client,
+        )
+        base_url = WatsonxOrchestrateTransformation.get_api_base_url(
+            cp4d_host, instance_id
+        )
+        auth_headers = {
+            "Authorization": f"Bearer {token}",
+            "Content-Type": "application/json",
+        }
+        text = WatsonxOrchestrateTransformation.extract_text_from_a2a_params(params)
+        body = WatsonxOrchestrateTransformation.build_wxo_run_body(
+            wxo_agent_id=wxo_agent_id, text=text, thread_id=thread_id
+        )
+
+        try:
+            response = await client.post(
+                f"{base_url}/v1/orchestrate/runs/stream",
+                json=body,
+                headers=auth_headers,
+                stream=True,
+            )
+            response.raise_for_status()
+            content_type = response.headers.get("content-type", "").lower()
+
+            if "text/event-stream" not in content_type:
+                response_body = await response.aread()
+                result = json.loads(response_body)
+                accumulated_text = (
+                    WatsonxOrchestrateTransformation.extract_text_from_wxo_result(
+                        result
+                    )
+                )
+            else:
+                accumulated_text = (
+                    await WatsonxOrchestrateHandler._accumulate_wxo_sse_text(response)
+                )
+
+            async for (
+                chunk
+            ) in WatsonxOrchestrateTransformation.fake_streaming_from_text(
+                text=accumulated_text,
+                request_id=request_id,
+                chunk_size=chunk_size,
+                delay_ms=delay_ms,
+            ):
+                yield chunk
+
+        except httpx.TransportError as exc:
+            verbose_logger.warning(
+                f"WXO: Streaming transport failed ({exc!r}), "
+                "falling back to non-streaming + fake streaming",
+                exc_info=True,
+            )
+            result = await WatsonxOrchestrateHandler.handle_non_streaming(
+                request_id=request_id,
+                params=params,
+                litellm_params=litellm_params,
+            )
+            response_text = (
+                WatsonxOrchestrateTransformation.extract_text_from_a2a_message_response(
+                    result
+                )
+            )
+            async for (
+                chunk
+            ) in WatsonxOrchestrateTransformation.fake_streaming_from_text(
+                text=response_text,
+                request_id=request_id,
+                chunk_size=chunk_size,
+                delay_ms=delay_ms,
+            ):
+                yield chunk

diff --git a/litellm/a2a_protocol/providers/watsonx_orchestrate/transformation.py b/litellm/a2a_protocol/providers/watsonx_orchestrate/transformation.py
new file mode 100644
--- /dev/null
+++ b/litellm/a2a_protocol/providers/watsonx_orchestrate/transformation.py
@@ -1,0 +1,224 @@
+"""
+Transformation layer for IBM watsonx Orchestrate (WXO) agent provider.
+
+WXO uses a REST API (not A2A/JSON-RPC) with an async-poll execution model:
+  POST /v1/orchestrate/runs       → submit run, get run_id
+  GET  /v1/orchestrate/runs/{id}  → poll until terminal state
+  POST /v1/orchestrate/runs/stream → native SSE streaming
+"""
+
+import asyncio
+from typing import Any, AsyncIterator, Dict, Optional
+from uuid import uuid4
+
+from litellm._logging import verbose_logger
+
+
+class WatsonxOrchestrateTransformation:
+    """
+    Handles request/response transformation between A2A and the WXO REST API.
+    """
+
+    TERMINAL_STATES = frozenset(
+        {"completed", "succeeded", "failed", "error", "cancelled"}
+    )
+    SUCCESS_STATES = frozenset({"completed", "succeeded"})
+
+    @staticmethod
+    def get_api_base_url(cp4d_host: str, instance_id: str) -> str:
+        """Build the WXO API base URL from host and instance ID."""
+        return f"{cp4d_host.rstrip('/')}/orchestrate/cpd/instances/{instance_id}"
+
+    @staticmethod
+    def extract_text_from_a2a_params(params: Dict[str, Any]) -> str:
+        """
+        Extract user message text from A2A MessageSendParams.
+
+        A2A format: params.message.parts[*] where part.kind == "text"
+        """
+        message = params.get("message", {})
+        parts = message.get("parts", [])
+        texts = []
+        for part in parts:
+            if not isinstance(part, dict):
+                continue
+            kind = part.get("kind")
+            if kind in (None, "", "text") and part.get("text"):
+                texts.append(part["text"])
+        return " ".join(texts) or ""
+
+    @staticmethod
+    def build_wxo_run_body(
+        wxo_agent_id: str,
+        text: str,
+        thread_id: Optional[str] = None,
+    ) -> Dict[str, Any]:
+        """Build the WXO POST /v1/orchestrate/runs request body."""
+        body: Dict[str, Any] = {
+            "agent_id": wxo_agent_id,
+            "message": {
+                "role": "user",
+                "content": [
+                    {
+                        "response_type": "conversational_search",
+                        "text": text,
+                    }
+                ],
+            },
+        }
+        if thread_id:
+            body["thread_id"] = thread_id
+        return body
+
+    @staticmethod
+    def extract_text_from_wxo_result(result: Any) -> str:
+        """
+        Extract response text from a WXO run result.
+
+        WXO can return text in several locations; checks in priority order per the API spec.
+        """
+        if not isinstance(result, dict):
+            return ""
+
+        # Primary: last_message.content[0].text
+        try:
+            text = result["last_message"]["content"][0]["text"]
+            if text:
+                return str(text)
+        except (KeyError, IndexError, TypeError):
+            pass
+
+        # Secondary: result.data.message.content[0].text
+        try:
+            text = result["result"]["data"]["message"]["content"][0]["text"]
+            if text:
+                return str(text)
+        except (KeyError, IndexError, TypeError):
+            pass
+
+        # Tertiary: results as a raw string
+        results = result.get("results")
+        if results and isinstance(results, str):
+            return results
+
+        return ""
+
+    @staticmethod
+    def extract_text_from_a2a_message_response(a2a_response: Dict[str, Any]) -> str:
+        result = a2a_response.get("result")
+        if not isinstance(result, dict):
+            verbose_logger.warning("WXO: A2A response missing result object")
+            return ""
+        parts = result.get("parts")
+        if not isinstance(parts, list):
+            verbose_logger.warning("WXO: A2A result has no parts list")
+            return ""
+        for part in parts:
+            if (
+                isinstance(part, dict)
+                and part.get("kind") == "text"
+                and part.get("text")
+            ):
+                return str(part["text"])
+        verbose_logger.warning("WXO: A2A result parts contained no text")
+        return ""
+
+    @staticmethod
+    def build_a2a_message_response(request_id: str, text: str) -> Dict[str, Any]:
+        """
+        Build a standard A2A non-streaming SendMessageResponse (kind=message).
+        """
+        return {
+            "jsonrpc": "2.0",
+            "id": request_id,
+            "result": {
+                "kind": "message",
+                "role": "agent",
+                "parts": [{"kind": "text", "text": text}],
+                "messageId": str(uuid4()),
+            },
+        }
+
+    @staticmethod
+    async def fake_streaming_from_text(
+        text: str,
+        request_id: str,
+        chunk_size: int = 50,
+        delay_ms: int = 10,
+    ) -> AsyncIterator[Dict[str, Any]]:
+        """
+        Emit standard A2A streaming events from a completed text response.
+
+        Event sequence:
+          1. task         (kind="task", state="submitted")
+          2. status-update (kind="status-update", state="working")
+          3. artifact-update chunks
+          4. status-update (kind="status-update", state="completed", final=True)
+        """
+        task_id = str(uuid4())
+        context_id = str(uuid4())
+        artifact_id = str(uuid4())
+
+        # 1. Task submitted
+        yield {
+            "jsonrpc": "2.0",
+            "id": request_id,
+            "result": {
+                "contextId": context_id,
+                "id": task_id,
+                "kind": "task",
+                "status": {"state": "submitted"},
+            },
+        }
+
+        # 2. Working
+        yield {
+            "jsonrpc": "2.0",
+            "id": request_id,
+            "result": {
+                "contextId": context_id,
+                "final": False,
+                "kind": "status-update",
+                "status": {"state": "working"},
+                "taskId": task_id,
+            },
+        }
+        await asyncio.sleep(delay_ms / 1000.0)
+
+        # 3. Artifact chunks (always emit at least one chunk, even for empty text)
+        text_to_chunk = text or ""
+        for i in range(0, max(len(text_to_chunk), 1), chunk_size):
+            chunk_text = text_to_chunk[i : i + chunk_size]
+            is_last = (i + chunk_size) >= max(len(text_to_chunk), 1)
+            yield {
+                "jsonrpc": "2.0",
+                "id": request_id,
+                "result": {
+                    "contextId": context_id,
+                    "kind": "artifact-update",
+                    "taskId": task_id,
+                    "artifact": {
+                        "artifactId": artifact_id,
+                        "parts": [{"kind": "text", "text": chunk_text}],
+                    },
+                },
+            }
+            if not is_last:
+                await asyncio.sleep(delay_ms / 1000.0)
+
+        # 4. Completed
+        yield {
+            "jsonrpc": "2.0",
+            "id": request_id,
+            "result": {
+                "contextId": context_id,
+                "final": True,
+                "kind": "status-update",
+                "status": {"state": "completed"},
+                "taskId": task_id,
+            },
+        }
+
+        verbose_logger.debug(
+            f"WXO: Fake streaming completed for request_id={request_id}"
+        )

diff --git a/litellm/proxy/public_endpoints/agent_create_fields.json b/litellm/proxy/public_endpoints/agent_create_fields.json
--- a/litellm/proxy/public_endpoints/agent_create_fields.json
+++ b/litellm/proxy/public_endpoints/agent_create_fields.json
@@ -189,6 +189,78 @@
     "litellm_params_template": {
       "custom_llm_provider": "vertex_ai"
     }
+  },
+  {
+    "agent_type": "watsonx_orchestrate",
+    "agent_type_display_name": "watsonx Orchestrate",
+    "description": "Connect to IBM watsonx Orchestrate agents via CP4D or IBM Cloud IAM",
+    "logo_url": "/ui/assets/logos/watsonx.svg",
+    "credential_fields": [
+      {
+        "key": "cp4d_host",
+        "label": "CP4D Host URL",
+        "placeholder": "https://cpd-cpd.apps.example.com",
+        "tooltip": "Your CP4D cluster base URL (e.g. https://cpd-cpd.apps.example.com). For IBM Cloud WXO, use the service endpoint.",
+        "required": true,
+        "field_type": "text",
+        "default_value": null,
+        "include_in_litellm_params": true
+      },
+      {
+        "key": "instance_id",
+        "label": "WXO Instance ID",
+        "placeholder": "1769134113217795",
+        "tooltip": "The numeric watsonx Orchestrate instance ID. Find it in the WXO service URL: /orchestrate/cpd/instances/<INSTANCE_ID>",
+        "required": true,
+        "field_type": "text",
+        "default_value": null,
+        "include_in_litellm_params": true
+      },
+      {
+        "key": "wxo_agent_id",
+        "label": "WXO Agent ID",
+        "placeholder": "588c8cdf-60f4-454b-8468-8702b19dca46",
+        "tooltip": "UUID of the agent in watsonx Orchestrate. Find it via the WXO console or GET /v1/orchestrate/agents.",
+        "required": true,
+        "field_type": "text",
+        "default_value": null,
+        "include_in_litellm_params": true
+      },
+      {
+        "key": "auth_mode",
+        "label": "Authentication Mode",
+        "placeholder": null,
+        "tooltip": "cp4d: on-prem / CloudPak for Data (requires username). ibm_cloud: IBM Cloud IAM (api_key only).",
+        "required": false,
+        "field_type": "select",
+        "options": ["cp4d", "ibm_cloud"],
+        "default_value": "cp4d",
+        "include_in_litellm_params": true
+      },
+      {
+        "key": "username",
+        "label": "Username (CP4D only)",
+        "placeholder": "admin",
+        "tooltip": "Your CP4D username. Required when auth_mode is 'cp4d'.",
+        "required": false,
+        "field_type": "text",
+        "default_value": null,
+        "include_in_litellm_params": true
+      },
+      {
+        "key": "api_key",
+        "label": "API Key",
+        "placeholder": null,
+        "tooltip": "CP4D API key (auth_mode=cp4d) or IBM Cloud API key (auth_mode=ibm_cloud).",
+        "required": true,
+        "field_type": "password",
+        "default_value": null,
+        "include_in_litellm_params": true
+      }
+    ],
+    "litellm_params_template": {
+      "custom_llm_provider": "watsonx_orchestrate"
+    }
   }
 ]
 

diff --git a/tests/test_litellm/a2a_protocol/providers/watsonx_orchestrate/test_watsonx_orchestrate_transformation.py b/tests/test_litellm/a2a_protocol/providers/watsonx_orchestrate/test_watsonx_orchestrate_transformation.py
new file mode 100644
--- /dev/null
+++ b/tests/test_litellm/a2a_protocol/providers/watsonx_orchestrate/test_watsonx_orchestrate_transformation.py
@@ -1,0 +1,265 @@
+import json
+import os
+import sys
+
+import pytest
+
+sys.path.insert(0, os.path.abspath("../../../../.."))
+
+from litellm.a2a_protocol.providers.config_manager import A2AProviderConfigManager
+from litellm.a2a_protocol.providers.watsonx_orchestrate.handler import (
+    WatsonxOrchestrateHandler,
+)
+from litellm.a2a_protocol.providers.watsonx_orchestrate.transformation import (
+    WatsonxOrchestrateTransformation,
+)
+
+
+class _JsonResponse:
+    def __init__(self, payload):
+        self.payload = payload
+
+    def raise_for_status(self):
+        pass
+
+    def json(self):
+        return self.payload
+
+
+class _ShortTtlTokenClient:
+    def __init__(self):
... diff truncated: showing 800 of 1035 lines

You can send follow-ups to the cloud agent here.

Comment thread litellm/a2a_protocol/providers/watsonx_orchestrate/handler.py
@Sameerlite

Copy link
Copy Markdown
Contributor Author

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.

Autofix Details

Bugbot Autofix prepared fixes for both issues found in the latest run.

  • ✅ Fixed: Streaming non-SSE path skips status check and polling
    • Added shared WXO run completion validation so non-SSE streaming JSON polls pending runs and raises on failed terminal states before extracting text.
  • ✅ Fixed: Default auth mode requires username marked optional in dashboard
    • Marked the dashboard username field required to match the default cp4d authentication mode runtime requirement.
Preview (4306dd1630)
diff --git a/litellm/a2a_protocol/providers/config_manager.py b/litellm/a2a_protocol/providers/config_manager.py
--- a/litellm/a2a_protocol/providers/config_manager.py
+++ b/litellm/a2a_protocol/providers/config_manager.py
@@ -48,4 +48,11 @@
 
             return BedrockAgentCoreA2AConfig()
 
+        if custom_llm_provider == "watsonx_orchestrate":
+            from litellm.a2a_protocol.providers.watsonx_orchestrate.config import (
+                WatsonxOrchestrateA2AConfig,
+            )
+
+            return WatsonxOrchestrateA2AConfig()
+
         return None

diff --git a/litellm/a2a_protocol/providers/watsonx_orchestrate/__init__.py b/litellm/a2a_protocol/providers/watsonx_orchestrate/__init__.py
new file mode 100644
--- /dev/null
+++ b/litellm/a2a_protocol/providers/watsonx_orchestrate/__init__.py
@@ -1,0 +1,3 @@
+"""
+IBM watsonx Orchestrate (WXO) A2A provider.
+"""

diff --git a/litellm/a2a_protocol/providers/watsonx_orchestrate/config.py b/litellm/a2a_protocol/providers/watsonx_orchestrate/config.py
new file mode 100644
--- /dev/null
+++ b/litellm/a2a_protocol/providers/watsonx_orchestrate/config.py
@@ -1,0 +1,55 @@
+"""
+A2A provider configuration for IBM watsonx Orchestrate (WXO).
+"""
+
+from typing import Any, AsyncIterator, Dict, Optional
+
+from litellm.a2a_protocol.providers.base import BaseA2AProviderConfig
+from litellm.a2a_protocol.providers.watsonx_orchestrate.handler import (
+    WatsonxOrchestrateHandler,
+)
+
+
+class WatsonxOrchestrateA2AConfig(BaseA2AProviderConfig):
+    """A2A bridge for IBM watsonx Orchestrate (REST runs API + poll/SSE)."""
+
+    async def handle_non_streaming(
+        self,
+        request_id: str,
+        params: Dict[str, Any],
+        api_base: Optional[str] = None,
+        **kwargs: Any,
+    ) -> Dict[str, Any]:
+        """Handle a non-streaming A2A request via WXO runs API."""
+        litellm_params = kwargs.get("litellm_params")
+        if not litellm_params:
+            raise ValueError(
+                "litellm_params is required for WatsonxOrchestrateA2AConfig "
+                "(must contain cp4d_host, instance_id, wxo_agent_id, api_key)"
+            )
+        return await WatsonxOrchestrateHandler.handle_non_streaming(
+            request_id=request_id,
+            params=params,
+            litellm_params=litellm_params,
+        )
+
+    async def handle_streaming(
+        self,
+        request_id: str,
+        params: Dict[str, Any],
+        api_base: Optional[str] = None,
+        **kwargs: Any,
+    ) -> AsyncIterator[Dict[str, Any]]:
+        """Handle a streaming A2A request via WXO streaming runs API."""
+        litellm_params = kwargs.get("litellm_params")
+        if not litellm_params:
+            raise ValueError(
+                "litellm_params is required for WatsonxOrchestrateA2AConfig "
+                "(must contain cp4d_host, instance_id, wxo_agent_id, api_key)"
+            )
+        async for chunk in WatsonxOrchestrateHandler.handle_streaming(
+            request_id=request_id,
+            params=params,
+            litellm_params=litellm_params,
+        ):
+            yield chunk

diff --git a/litellm/a2a_protocol/providers/watsonx_orchestrate/handler.py b/litellm/a2a_protocol/providers/watsonx_orchestrate/handler.py
new file mode 100644
--- /dev/null
+++ b/litellm/a2a_protocol/providers/watsonx_orchestrate/handler.py
@@ -1,0 +1,382 @@
+"""
+Handler for IBM watsonx Orchestrate (WXO) agent provider.
+"""
+
+import asyncio
+import hashlib
+import json
+import time
+from typing import Any, AsyncIterator, Dict, Optional, Tuple, cast
+
+import httpx
+
+from litellm._logging import verbose_logger
+from litellm.a2a_protocol.providers.watsonx_orchestrate.transformation import (
+    WatsonxOrchestrateTransformation,
+)
+from litellm.llms.custom_httpx.http_handler import (
+    AsyncHTTPHandler,
+    get_async_httpx_client,
+)
+from litellm.types.llms.custom_http import httpxSpecialProvider
+
+_IBM_CLOUD_IAM_URL = "https://iam.cloud.ibm.com/identity/token"
+_POLL_INTERVAL_S = 2.0
+_MAX_POLL_ATTEMPTS = 90
+_TOKEN_CACHE_TTL_BUFFER_S = 60
+_token_cache: Dict[str, Tuple[str, float]] = {}
+
+
+class WatsonxOrchestrateHandler:
+    @staticmethod
+    def _http_client(timeout: float = 90.0) -> AsyncHTTPHandler:
+        return get_async_httpx_client(
+            llm_provider=cast(Any, httpxSpecialProvider.A2AProvider),
+            params={"timeout": timeout},
+        )
+
+    @staticmethod
+    def _token_cache_key(
+        auth_mode: str,
+        cp4d_host: str,
+        api_key: str,
+        username: Optional[str],
+    ) -> str:
+        material = f"{auth_mode}:{cp4d_host}:{username or ''}:{api_key}"
+        return hashlib.sha256(material.encode()).hexdigest()
+
+    @staticmethod
+    def _cp4d_token_ttl_seconds(
+        expiration: Any, now_wall: Optional[float] = None
+    ) -> int:
+        # CP4D returns expiration as absolute Unix epoch seconds, not a duration.
+        expires_at = int(expiration)
+        wall = now_wall if now_wall is not None else time.time()
+        return max(expires_at - int(wall), 0)
+
+    @staticmethod
+    async def _get_bearer_token(
+        cp4d_host: str,
+        auth_mode: str,
+        api_key: str,
+        username: Optional[str] = None,
+        client: Optional[AsyncHTTPHandler] = None,
+    ) -> str:
+        cache_key = WatsonxOrchestrateHandler._token_cache_key(
+            auth_mode, cp4d_host, api_key, username
+        )
+        now = time.monotonic()
+        cached = _token_cache.get(cache_key)
+        if cached and cached[1] > now:
+            return cached[0]
+
+        if client is None:
+            client = WatsonxOrchestrateHandler._http_client(timeout=30.0)
+
+        if auth_mode == "ibm_cloud":
+            response = await client.post(
+                _IBM_CLOUD_IAM_URL,
+                data={
+                    "grant_type": "urn:ibm:params:oauth:grant-type:apikey",
+                    "apikey": api_key,
+                },
+                headers={"Content-Type": "application/x-www-form-urlencoded"},
+            )
+            response.raise_for_status()
+            payload = response.json()
+            token = str(payload["access_token"])
+            ttl_s = int(payload.get("expires_in", 3600))
+        else:
+            if not username:
+                raise ValueError(
+                    "'username' is required in litellm_params when auth_mode='cp4d'"
+                )
+            token_url = f"{cp4d_host.rstrip('/')}/icp4d-api/v1/authorize"
+            response = await client.post(
+                token_url,
+                json={"username": username, "api_key": api_key},
+                headers={"Content-Type": "application/json"},
+            )
+            response.raise_for_status()
+            payload = response.json()
+            token = str(payload["token"])
+            expiration = payload.get("expiration")
+            if expiration is None:
+                ttl_s = 3600
+            else:
+                ttl_s = WatsonxOrchestrateHandler._cp4d_token_ttl_seconds(expiration)
+
+        expires_at = now + max(ttl_s - _TOKEN_CACHE_TTL_BUFFER_S, 0)
+        _token_cache[cache_key] = (token, expires_at)
+        return token
+
+    @staticmethod
+    async def _poll_run(
+        base_url: str,
+        run_id: str,
+        auth_headers: Dict[str, str],
+        client: AsyncHTTPHandler,
+        max_attempts: int = _MAX_POLL_ATTEMPTS,
+        interval_s: float = _POLL_INTERVAL_S,
+    ) -> Dict[str, Any]:
+        url = f"{base_url}/v1/orchestrate/runs/{run_id}"
+
+        for attempt in range(max_attempts):
+            await asyncio.sleep(interval_s)
+            response = await client.get(url, headers=auth_headers)
+            response.raise_for_status()
+            result: Dict[str, Any] = response.json()
+            status = result.get("status", "")
+            verbose_logger.debug(
+                f"WXO: Poll {attempt + 1}/{max_attempts} run='{run_id}' status='{status}'"
+            )
+            if status in WatsonxOrchestrateTransformation.TERMINAL_STATES:
+                return result
+
+        raise TimeoutError(
+            f"WXO run '{run_id}' did not reach a terminal state after "
+            f"{max_attempts * interval_s:.0f}s"
+        )
+
+    @staticmethod
+    async def _get_successful_run_data(
+        run_data: Dict[str, Any],
+        base_url: str,
+        auth_headers: Dict[str, str],
+        client: AsyncHTTPHandler,
+    ) -> Dict[str, Any]:
+        status = run_data.get("status", "")
+        if status not in WatsonxOrchestrateTransformation.TERMINAL_STATES:
+            run_id = run_data.get("run_id") or run_data.get("id") or ""
+            if not run_id:
+                raise ValueError(f"WXO: No run_id in response: {run_data}")
+            run_data = await WatsonxOrchestrateHandler._poll_run(
+                base_url=base_url,
+                run_id=run_id,
+                auth_headers=auth_headers,
+                client=client,
+            )
+            status = run_data.get("status", "")
+
+        if status not in WatsonxOrchestrateTransformation.SUCCESS_STATES:
+            raise RuntimeError(
+                f"WXO run ended with non-success status '{status}': {run_data}"
+            )
+
+        return run_data
+
+    @staticmethod
+    async def _accumulate_wxo_sse_text(response: Any) -> str:
+        accumulated_text = ""
+        async for line in response.aiter_lines():
+            if not line.startswith("data:"):
+                continue
+            data_str = line[5:].strip()
+            if not data_str or data_str == "[DONE]":
+                continue
+            try:
+                event = json.loads(data_str)
+            except json.JSONDecodeError:
+                continue
+            chunk_text = WatsonxOrchestrateTransformation.extract_text_from_wxo_result(
+                event
+            )
+            if chunk_text:
+                accumulated_text += chunk_text
+        return accumulated_text
+
+    @staticmethod
+    def _extract_litellm_params(litellm_params: Dict[str, Any]) -> tuple:
+        cp4d_host = litellm_params.get("cp4d_host") or ""
+        instance_id = litellm_params.get("instance_id") or ""
+        wxo_agent_id = litellm_params.get("wxo_agent_id") or ""
+        api_key = litellm_params.get("api_key") or ""
+        username = litellm_params.get("username") or None
+        auth_mode = litellm_params.get("auth_mode") or "cp4d"
+        thread_id = litellm_params.get("thread_id") or None
+
+        if not cp4d_host:
+            raise ValueError("'cp4d_host' is required in litellm_params for WXO agents")
+        if not instance_id:
+            raise ValueError(
+                "'instance_id' is required in litellm_params for WXO agents"
+            )
+        if not wxo_agent_id:
+            raise ValueError(
+                "'wxo_agent_id' is required in litellm_params for WXO agents"
+            )
+        if not api_key:
+            raise ValueError("'api_key' is required in litellm_params for WXO agents")
+
+        return (
+            cp4d_host,
+            instance_id,
+            wxo_agent_id,
+            api_key,
+            username,
+            auth_mode,
+            thread_id,
+        )
+
+    @staticmethod
+    async def handle_non_streaming(
+        request_id: str,
+        params: Dict[str, Any],
+        litellm_params: Dict[str, Any],
+    ) -> Dict[str, Any]:
+        (
+            cp4d_host,
+            instance_id,
+            wxo_agent_id,
+            api_key,
+            username,
+            auth_mode,
+            thread_id,
+        ) = WatsonxOrchestrateHandler._extract_litellm_params(litellm_params)
+
+        client = WatsonxOrchestrateHandler._http_client(timeout=90.0)
+        token = await WatsonxOrchestrateHandler._get_bearer_token(
+            cp4d_host=cp4d_host,
+            auth_mode=auth_mode,
+            api_key=api_key,
+            username=username,
+            client=client,
+        )
+        base_url = WatsonxOrchestrateTransformation.get_api_base_url(
+            cp4d_host, instance_id
+        )
+        auth_headers = {
+            "Authorization": f"Bearer {token}",
+            "Content-Type": "application/json",
+            "Accept": "application/json",
+        }
+
+        text = WatsonxOrchestrateTransformation.extract_text_from_a2a_params(params)
+        body = WatsonxOrchestrateTransformation.build_wxo_run_body(
+            wxo_agent_id=wxo_agent_id, text=text, thread_id=thread_id
+        )
+
+        run_response = await client.post(
+            f"{base_url}/v1/orchestrate/runs",
+            json=body,
+            headers=auth_headers,
+        )
+        run_response.raise_for_status()
+        run_data: Dict[str, Any] = run_response.json()
+
+        run_data = await WatsonxOrchestrateHandler._get_successful_run_data(
+            run_data=run_data,
+            base_url=base_url,
+            auth_headers=auth_headers,
+            client=client,
+        )
+
+        response_text = WatsonxOrchestrateTransformation.extract_text_from_wxo_result(
+            run_data
+        )
+        return WatsonxOrchestrateTransformation.build_a2a_message_response(
+            request_id=request_id, text=response_text
+        )
+
+    @staticmethod
+    async def handle_streaming(
+        request_id: str,
+        params: Dict[str, Any],
+        litellm_params: Dict[str, Any],
+        chunk_size: int = 50,
+        delay_ms: int = 10,
+    ) -> AsyncIterator[Dict[str, Any]]:
+        (
+            cp4d_host,
+            instance_id,
+            wxo_agent_id,
+            api_key,
+            username,
+            auth_mode,
+            thread_id,
+        ) = WatsonxOrchestrateHandler._extract_litellm_params(litellm_params)
+
+        client = WatsonxOrchestrateHandler._http_client(timeout=120.0)
+        token = await WatsonxOrchestrateHandler._get_bearer_token(
+            cp4d_host=cp4d_host,
+            auth_mode=auth_mode,
+            api_key=api_key,
+            username=username,
+            client=client,
+        )
+        base_url = WatsonxOrchestrateTransformation.get_api_base_url(
+            cp4d_host, instance_id
+        )
+        auth_headers = {
+            "Authorization": f"Bearer {token}",
+            "Content-Type": "application/json",
+        }
+        text = WatsonxOrchestrateTransformation.extract_text_from_a2a_params(params)
+        body = WatsonxOrchestrateTransformation.build_wxo_run_body(
+            wxo_agent_id=wxo_agent_id, text=text, thread_id=thread_id
+        )
+
+        try:
+            response = await client.post(
+                f"{base_url}/v1/orchestrate/runs/stream",
+                json=body,
+                headers=auth_headers,
+                stream=True,
+            )
+            response.raise_for_status()
+            content_type = response.headers.get("content-type", "").lower()
+
+            if "text/event-stream" not in content_type:
+                response_body = await response.aread()
+                result = json.loads(response_body)
+                result = await WatsonxOrchestrateHandler._get_successful_run_data(
+                    run_data=result,
+                    base_url=base_url,
+                    auth_headers=auth_headers,
+                    client=client,
+                )
+                accumulated_text = (
+                    WatsonxOrchestrateTransformation.extract_text_from_wxo_result(
+                        result
+                    )
+                )
+            else:
+                accumulated_text = (
+                    await WatsonxOrchestrateHandler._accumulate_wxo_sse_text(response)
+                )
+
+            async for (
+                chunk
+            ) in WatsonxOrchestrateTransformation.fake_streaming_from_text(
+                text=accumulated_text,
+                request_id=request_id,
+                chunk_size=chunk_size,
+                delay_ms=delay_ms,
+            ):
+                yield chunk
+
+        except httpx.TransportError as exc:
+            verbose_logger.warning(
+                f"WXO: Streaming transport failed ({exc!r}), "
+                "falling back to non-streaming + fake streaming",
+                exc_info=True,
+            )
+            result = await WatsonxOrchestrateHandler.handle_non_streaming(
+                request_id=request_id,
+                params=params,
+                litellm_params=litellm_params,
+            )
+            response_text = (
+                WatsonxOrchestrateTransformation.extract_text_from_a2a_message_response(
+                    result
+                )
+            )
+            async for (
+                chunk
+            ) in WatsonxOrchestrateTransformation.fake_streaming_from_text(
+                text=response_text,
+                request_id=request_id,
+                chunk_size=chunk_size,
+                delay_ms=delay_ms,
+            ):
+                yield chunk

diff --git a/litellm/a2a_protocol/providers/watsonx_orchestrate/transformation.py b/litellm/a2a_protocol/providers/watsonx_orchestrate/transformation.py
new file mode 100644
--- /dev/null
+++ b/litellm/a2a_protocol/providers/watsonx_orchestrate/transformation.py
@@ -1,0 +1,224 @@
+"""
+Transformation layer for IBM watsonx Orchestrate (WXO) agent provider.
+
+WXO uses a REST API (not A2A/JSON-RPC) with an async-poll execution model:
+  POST /v1/orchestrate/runs       → submit run, get run_id
+  GET  /v1/orchestrate/runs/{id}  → poll until terminal state
+  POST /v1/orchestrate/runs/stream → native SSE streaming
+"""
+
+import asyncio
+from typing import Any, AsyncIterator, Dict, Optional
+from uuid import uuid4
+
+from litellm._logging import verbose_logger
+
+
+class WatsonxOrchestrateTransformation:
+    """
+    Handles request/response transformation between A2A and the WXO REST API.
+    """
+
+    TERMINAL_STATES = frozenset(
+        {"completed", "succeeded", "failed", "error", "cancelled"}
+    )
+    SUCCESS_STATES = frozenset({"completed", "succeeded"})
+
+    @staticmethod
+    def get_api_base_url(cp4d_host: str, instance_id: str) -> str:
+        """Build the WXO API base URL from host and instance ID."""
+        return f"{cp4d_host.rstrip('/')}/orchestrate/cpd/instances/{instance_id}"
+
+    @staticmethod
+    def extract_text_from_a2a_params(params: Dict[str, Any]) -> str:
+        """
+        Extract user message text from A2A MessageSendParams.
+
+        A2A format: params.message.parts[*] where part.kind == "text"
+        """
+        message = params.get("message", {})
+        parts = message.get("parts", [])
+        texts = []
+        for part in parts:
+            if not isinstance(part, dict):
+                continue
+            kind = part.get("kind")
+            if kind in (None, "", "text") and part.get("text"):
+                texts.append(part["text"])
+        return " ".join(texts) or ""
+
+    @staticmethod
+    def build_wxo_run_body(
+        wxo_agent_id: str,
+        text: str,
+        thread_id: Optional[str] = None,
+    ) -> Dict[str, Any]:
+        """Build the WXO POST /v1/orchestrate/runs request body."""
+        body: Dict[str, Any] = {
+            "agent_id": wxo_agent_id,
+            "message": {
+                "role": "user",
+                "content": [
+                    {
+                        "response_type": "conversational_search",
+                        "text": text,
+                    }
+                ],
+            },
+        }
+        if thread_id:
+            body["thread_id"] = thread_id
+        return body
+
+    @staticmethod
+    def extract_text_from_wxo_result(result: Any) -> str:
+        """
+        Extract response text from a WXO run result.
+
+        WXO can return text in several locations; checks in priority order per the API spec.
+        """
+        if not isinstance(result, dict):
+            return ""
+
+        # Primary: last_message.content[0].text
+        try:
+            text = result["last_message"]["content"][0]["text"]
+            if text:
+                return str(text)
+        except (KeyError, IndexError, TypeError):
+            pass
+
+        # Secondary: result.data.message.content[0].text
+        try:
+            text = result["result"]["data"]["message"]["content"][0]["text"]
+            if text:
+                return str(text)
+        except (KeyError, IndexError, TypeError):
+            pass
+
+        # Tertiary: results as a raw string
+        results = result.get("results")
+        if results and isinstance(results, str):
+            return results
+
+        return ""
+
+    @staticmethod
+    def extract_text_from_a2a_message_response(a2a_response: Dict[str, Any]) -> str:
+        result = a2a_response.get("result")
+        if not isinstance(result, dict):
+            verbose_logger.warning("WXO: A2A response missing result object")
+            return ""
+        parts = result.get("parts")
+        if not isinstance(parts, list):
+            verbose_logger.warning("WXO: A2A result has no parts list")
+            return ""
+        for part in parts:
+            if (
+                isinstance(part, dict)
+                and part.get("kind") == "text"
+                and part.get("text")
+            ):
+                return str(part["text"])
+        verbose_logger.warning("WXO: A2A result parts contained no text")
+        return ""
+
+    @staticmethod
+    def build_a2a_message_response(request_id: str, text: str) -> Dict[str, Any]:
+        """
+        Build a standard A2A non-streaming SendMessageResponse (kind=message).
+        """
+        return {
+            "jsonrpc": "2.0",
+            "id": request_id,
+            "result": {
+                "kind": "message",
+                "role": "agent",
+                "parts": [{"kind": "text", "text": text}],
+                "messageId": str(uuid4()),
+            },
+        }
+
+    @staticmethod
+    async def fake_streaming_from_text(
+        text: str,
+        request_id: str,
+        chunk_size: int = 50,
+        delay_ms: int = 10,
+    ) -> AsyncIterator[Dict[str, Any]]:
+        """
+        Emit standard A2A streaming events from a completed text response.
+
+        Event sequence:
+          1. task         (kind="task", state="submitted")
+          2. status-update (kind="status-update", state="working")
+          3. artifact-update chunks
+          4. status-update (kind="status-update", state="completed", final=True)
+        """
+        task_id = str(uuid4())
+        context_id = str(uuid4())
+        artifact_id = str(uuid4())
+
+        # 1. Task submitted
+        yield {
+            "jsonrpc": "2.0",
+            "id": request_id,
+            "result": {
+                "contextId": context_id,
+                "id": task_id,
+                "kind": "task",
+                "status": {"state": "submitted"},
+            },
+        }
+
+        # 2. Working
+        yield {
+            "jsonrpc": "2.0",
+            "id": request_id,
+            "result": {
+                "contextId": context_id,
+                "final": False,
+                "kind": "status-update",
+                "status": {"state": "working"},
+                "taskId": task_id,
+            },
+        }
+        await asyncio.sleep(delay_ms / 1000.0)
+
+        # 3. Artifact chunks (always emit at least one chunk, even for empty text)
+        text_to_chunk = text or ""
+        for i in range(0, max(len(text_to_chunk), 1), chunk_size):
+            chunk_text = text_to_chunk[i : i + chunk_size]
+            is_last = (i + chunk_size) >= max(len(text_to_chunk), 1)
+            yield {
+                "jsonrpc": "2.0",
+                "id": request_id,
+                "result": {
+                    "contextId": context_id,
+                    "kind": "artifact-update",
+                    "taskId": task_id,
+                    "artifact": {
+                        "artifactId": artifact_id,
+                        "parts": [{"kind": "text", "text": chunk_text}],
+                    },
+                },
+            }
+            if not is_last:
+                await asyncio.sleep(delay_ms / 1000.0)
+
+        # 4. Completed
+        yield {
+            "jsonrpc": "2.0",
+            "id": request_id,
+            "result": {
+                "contextId": context_id,
+                "final": True,
+                "kind": "status-update",
+                "status": {"state": "completed"},
+                "taskId": task_id,
+            },
+        }
+
+        verbose_logger.debug(
+            f"WXO: Fake streaming completed for request_id={request_id}"
+        )

diff --git a/litellm/proxy/public_endpoints/agent_create_fields.json b/litellm/proxy/public_endpoints/agent_create_fields.json
--- a/litellm/proxy/public_endpoints/agent_create_fields.json
+++ b/litellm/proxy/public_endpoints/agent_create_fields.json
@@ -189,6 +189,78 @@
     "litellm_params_template": {
       "custom_llm_provider": "vertex_ai"
     }
+  },
+  {
+    "agent_type": "watsonx_orchestrate",
+    "agent_type_display_name": "watsonx Orchestrate",
+    "description": "Connect to IBM watsonx Orchestrate agents via CP4D or IBM Cloud IAM",
+    "logo_url": "/ui/assets/logos/watsonx.svg",
+    "credential_fields": [
+      {
+        "key": "cp4d_host",
+        "label": "CP4D Host URL",
+        "placeholder": "https://cpd-cpd.apps.example.com",
+        "tooltip": "Your CP4D cluster base URL (e.g. https://cpd-cpd.apps.example.com). For IBM Cloud WXO, use the service endpoint.",
+        "required": true,
+        "field_type": "text",
+        "default_value": null,
+        "include_in_litellm_params": true
+      },
+      {
+        "key": "instance_id",
+        "label": "WXO Instance ID",
+        "placeholder": "1769134113217795",
+        "tooltip": "The numeric watsonx Orchestrate instance ID. Find it in the WXO service URL: /orchestrate/cpd/instances/<INSTANCE_ID>",
+        "required": true,
+        "field_type": "text",
+        "default_value": null,
+        "include_in_litellm_params": true
+      },
+      {
+        "key": "wxo_agent_id",
+        "label": "WXO Agent ID",
+        "placeholder": "588c8cdf-60f4-454b-8468-8702b19dca46",
+        "tooltip": "UUID of the agent in watsonx Orchestrate. Find it via the WXO console or GET /v1/orchestrate/agents.",
+        "required": true,
+        "field_type": "text",
+        "default_value": null,
+        "include_in_litellm_params": true
+      },
+      {
+        "key": "auth_mode",
+        "label": "Authentication Mode",
+        "placeholder": null,
+        "tooltip": "cp4d: on-prem / CloudPak for Data (requires username). ibm_cloud: IBM Cloud IAM (api_key only).",
+        "required": false,
+        "field_type": "select",
+        "options": ["cp4d", "ibm_cloud"],
+        "default_value": "cp4d",
+        "include_in_litellm_params": true
+      },
+      {
+        "key": "username",
+        "label": "Username (CP4D only)",
+        "placeholder": "admin",
+        "tooltip": "Your CP4D username. Required when auth_mode is 'cp4d'.",
+        "required": true,
+        "field_type": "text",
+        "default_value": null,
+        "include_in_litellm_params": true
+      },
+      {
+        "key": "api_key",
+        "label": "API Key",
+        "placeholder": null,
+        "tooltip": "CP4D API key (auth_mode=cp4d) or IBM Cloud API key (auth_mode=ibm_cloud).",
+        "required": true,
+        "field_type": "password",
+        "default_value": null,
+        "include_in_litellm_params": true
+      }
+    ],
+    "litellm_params_template": {
+      "custom_llm_provider": "watsonx_orchestrate"
+    }
   }
 ]
 

diff --git a/tests/test_litellm/a2a_protocol/providers/watsonx_orchestrate/test_watsonx_orchestrate_transformation.py b/tests/test_litellm/a2a_protocol/providers/watsonx_orchestrate/test_watsonx_orchestrate_transformation.py
new file mode 100644
--- /dev/null
+++ b/tests/test_litellm/a2a_protocol/providers/watsonx_orchestrate/test_watsonx_orchestrate_transformation.py
@@ -1,0 +1,393 @@
+import json
+import os
+import sys
+from pathlib import Path
+
+import pytest
+
+sys.path.insert(0, os.path.abspath("../../../../.."))
... diff truncated: showing 800 of 1185 lines

You can send follow-ups to the cloud agent here.

Comment thread litellm/a2a_protocol/providers/watsonx_orchestrate/handler.py Outdated
Comment thread litellm/proxy/public_endpoints/agent_create_fields.json
@Sameerlite

Copy link
Copy Markdown
Contributor Author

bugbot run

Comment thread litellm/proxy/public_endpoints/agent_create_fields.json
Username is only required for cp4d auth; ibm_cloud uses api_key alone.
Backend still validates username when auth_mode is cp4d.

Co-authored-by: Cursor <cursoragent@cursor.com>
@Sameerlite

Copy link
Copy Markdown
Contributor Author

@greptileai

@Sameerlite

Copy link
Copy Markdown
Contributor Author

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit e034cec. Configure here.

Username is not required in agent_create_fields.json; backend validates
for cp4d auth_mode only.

Co-authored-by: Cursor <cursoragent@cursor.com>
@Sameerlite

Copy link
Copy Markdown
Contributor Author

@greptileai

@Sameerlite

Copy link
Copy Markdown
Contributor Author

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 900926a. Configure here.

@Sameerlite
Sameerlite requested a review from mateo-berri June 1, 2026 13:38
@mateo-berri

Copy link
Copy Markdown
Contributor

@greptileai

@mateo-berri

Copy link
Copy Markdown
Contributor

@greptileai

Narrow the httpx.TransportError fallback in handle_streaming so it only
covers the initial POST /runs/stream. Errors during polling or SSE
consumption now propagate instead of triggering handle_non_streaming,
which would have submitted a duplicate WXO run for the same request.
@mateo-berri

Copy link
Copy Markdown
Contributor

@greptileai

Return a typed WXORequestParams NamedTuple from _extract_litellm_params
instead of a positional tuple so call sites read params by name, and send
the user message with response_type 'text' so the run body is valid across
all WXO agent configurations rather than the search-specific type.
@mateo-berri

Copy link
Copy Markdown
Contributor

@greptileai

@mateo-berri

Copy link
Copy Markdown
Contributor

@greptileai

@mateo-berri mateo-berri left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM; thanks!

@Sameerlite
Sameerlite merged commit c8bcfbb into litellm_internal_staging Jun 2, 2026
119 checks passed
@Sameerlite
Sameerlite deleted the litellm_watsonx_orchestrate_agents branch June 2, 2026 13:11
dan2k3k4 pushed a commit to dan2k3k4/litellm that referenced this pull request Jun 2, 2026
* feat(a2a): add watsonx Orchestrate agent provider

Bridge A2A message/send to WXO runs API (CP4D and IBM Cloud IAM auth),
with dashboard agent type metadata and unit tests for transformations.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(a2a): use shared httpx client and cache WXO auth tokens

Route WXO streaming through get_async_httpx_client (TLS verification
enabled). Cache bearer tokens with TTL buffer. Extract A2A reply text via
a dedicated helper instead of hard-coded JSON paths.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(a2a): treat CP4D token expiration as absolute Unix time

CP4D /authorize returns expiration as epoch seconds, not TTL. Compute
remaining lifetime against wall clock so cached tokens refresh before expiry.

Co-authored-by: Cursor <cursoragent@cursor.com>

* style(a2a): black-format watsonx orchestrate handler for CI py312

Co-authored-by: Cursor <cursoragent@cursor.com>

* Fix watsonx orchestrate edge cases

* Fix WXO streaming fallback error handling

* Fix watsonx orchestrate run completion handling

* fix(a2a): make WXO username optional in agent create UI

Username is only required for cp4d auth; ibm_cloud uses api_key alone.
Backend still validates username when auth_mode is cp4d.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(a2a): align WXO dashboard field test with optional username

Username is not required in agent_create_fields.json; backend validates
for cp4d auth_mode only.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(a2a): send Accept header on WXO streaming run request

* fix(a2a/wxo): scope streaming transport fallback to initial POST only

Narrow the httpx.TransportError fallback in handle_streaming so it only
covers the initial POST /runs/stream. Errors during polling or SSE
consumption now propagate instead of triggering handle_non_streaming,
which would have submitted a duplicate WXO run for the same request.

* refactor(a2a/wxo): type run-param extraction and use text response_type

Return a typed WXORequestParams NamedTuple from _extract_litellm_params
instead of a positional tuple so call sites read params by name, and send
the user message with response_type 'text' so the run body is valid across
all WXO agent configurations rather than the search-specific type.

* fix(a2a/wxo): evict expired token cache entries and raise asyncio.TimeoutError on poll timeout

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
hbjydev pushed a commit to hbjydev/phoebe that referenced this pull request Jun 14, 2026
…9.0) (#93)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [ghcr.io/berriai/litellm](https://images.chainguard.dev/directory/image/wolfi-base/overview) ([source](https://github.com/BerriAI/litellm)) | minor | `v1.88.1` → `v1.89.0` |

---

### Release Notes

<details>
<summary>BerriAI/litellm (ghcr.io/berriai/litellm)</summary>

### [`v1.89.0`](https://github.com/BerriAI/litellm/releases/tag/v1.89.0)

[Compare Source](https://github.com/BerriAI/litellm/compare/v1.89.0...v1.89.0)

##### Verify Docker Image Signature

All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0).

**Verify using the pinned commit hash (recommended):**

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

```bash
cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
  ghcr.io/berriai/litellm:v1.89.0
```

**Verify using the release tag (convenience):**

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

```bash
cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/v1.89.0/cosign.pub \
  ghcr.io/berriai/litellm:v1.89.0
```

Expected output:

```
The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key
```

***

##### What's Changed

- test(responses): bump deprecated gemini-3-pro-preview to gemini-3.1-pro-preview by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29433](https://github.com/BerriAI/litellm/pull/29433)
- fix: map mistral/ministral-8b-latest in model price map by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29453](https://github.com/BerriAI/litellm/pull/29453)
- fix(datadog): split oversized batches on 413 instead of re-queueing forever by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29444](https://github.com/BerriAI/litellm/pull/29444)
- feat(otel): allowlist team\_metadata sub-keys promoted to baggage by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29442](https://github.com/BerriAI/litellm/pull/29442)
- fix: stop use\_chat\_completions\_api flag from leaking into provider request body by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29447](https://github.com/BerriAI/litellm/pull/29447)
- fix(anthropic, fireworks): inline legacy $ref defs in tool schemas by [@&#8203;milan-berri](https://github.com/milan-berri) in [#&#8203;28646](https://github.com/BerriAI/litellm/pull/28646)
- fix(proxy): omit OpenAI \[DONE] on google-genai streamGenerateContent by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29426](https://github.com/BerriAI/litellm/pull/29426)
- ci(release): create stable/X.Y.x line branch on X.Y.0 tags by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29457](https://github.com/BerriAI/litellm/pull/29457)
- fix(vector-stores): support engines URL for Vertex AI Search by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;27885](https://github.com/BerriAI/litellm/pull/27885)
- fix(ui): render caller-supplied filter options in caller order by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29462](https://github.com/BerriAI/litellm/pull/29462)
- fix(batches): skip unnecessary batch input file reads by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29114](https://github.com/BerriAI/litellm/pull/29114)
- docs(agents): clarify when to create new test files by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29472](https://github.com/BerriAI/litellm/pull/29472)
- Litellm OSS Staging by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29161](https://github.com/BerriAI/litellm/pull/29161)
- fix(mcp): clear allowed\_tools and tool overrides on MCP server edit by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29411](https://github.com/BerriAI/litellm/pull/29411)
- Litellm OSS Staging 010626 by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29422](https://github.com/BerriAI/litellm/pull/29422)
- fix(ci): make CircleCI rerun-failed-tests collect tests when 2+ test files fail by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29475](https://github.com/BerriAI/litellm/pull/29475)
- feat(a2a): watsonx Orchestrate agent provider by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29410](https://github.com/BerriAI/litellm/pull/29410)
- fix(azure\_ai): strip tool-level extra fields on 400 and retry by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29479](https://github.com/BerriAI/litellm/pull/29479)
- fix(docs): remove fixed dimensions from README hero image by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29496](https://github.com/BerriAI/litellm/pull/29496)
- Litellm oss staging by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29492](https://github.com/BerriAI/litellm/pull/29492)
- fix: small CLAUDE.md nits by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29504](https://github.com/BerriAI/litellm/pull/29504)
- Add MCP semantic conventions to otelv2 by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29468](https://github.com/BerriAI/litellm/pull/29468)
- fix(passthrough): emit otel guardrail span when a guardrail blocks by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29470](https://github.com/BerriAI/litellm/pull/29470)
- fix(proxy): strip NUL bytes from spend log payloads to prevent PostgreSQL 22P05 by [@&#8203;milan-berri](https://github.com/milan-berri) in [#&#8203;29515](https://github.com/BerriAI/litellm/pull/29515)
- \[internal copy of [#&#8203;28008](https://github.com/BerriAI/litellm/issues/28008)] Support MCP OAuth passthrough and issuer-scoped JWT auth by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;28356](https://github.com/BerriAI/litellm/pull/28356)
- feat(vector-stores): forward per-request params to Vertex AI Search by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29459](https://github.com/BerriAI/litellm/pull/29459)
- feat(proxy): add per-MCP-server RPM rate limiting for keys and teams by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29482](https://github.com/BerriAI/litellm/pull/29482)
- fix(tests): drop module-level test calls that break local\_testing collection by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29520](https://github.com/BerriAI/litellm/pull/29520)
- feat(agents): add LangFlow agent provider with A2A session bridging by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28963](https://github.com/BerriAI/litellm/pull/28963)
- fix(ui/agents): make A2A skill tags enterable and validated by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29512](https://github.com/BerriAI/litellm/pull/29512)
- \[internal copy of [#&#8203;29232](https://github.com/BerriAI/litellm/issues/29232)] feat: route future Claude models to Anthropic provider via pattern matching by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29239](https://github.com/BerriAI/litellm/pull/29239)
- fix(tests): drop import-time completion call in test\_register\_model by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29521](https://github.com/BerriAI/litellm/pull/29521)
- test: stabilize batch VCR coverage and stop live upload/network leaks by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29477](https://github.com/BerriAI/litellm/pull/29477)
- \[internal copy of [#&#8203;29003](https://github.com/BerriAI/litellm/issues/29003)] fix(vertex\_ai): use user-supplied api\_base as is for Model Garden OpenAI-compat path by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29530](https://github.com/BerriAI/litellm/pull/29530)
- feat(proxy): native /health/drain preStop hook for graceful shutdown by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29439](https://github.com/BerriAI/litellm/pull/29439)
- fix(auth): preserve 401 status for expired JWTs in OTel traces by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29510](https://github.com/BerriAI/litellm/pull/29510)
- fix(otel): capture 401 error details in management endpoint spans by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29535](https://github.com/BerriAI/litellm/pull/29535)
- test(proxy/utils): pin bottom-of-file helper behavior by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29509](https://github.com/BerriAI/litellm/pull/29509)
- test(proxy/utils): pin PrismaClient and spend-update behavior by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29488](https://github.com/BerriAI/litellm/pull/29488)
- test(proxy/utils): pin ProxyLogging behavior by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29485](https://github.com/BerriAI/litellm/pull/29485)
- fix: missing span for guardrail passthrough by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29552](https://github.com/BerriAI/litellm/pull/29552)
- fix(auth): let internal users view search tools by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29542](https://github.com/BerriAI/litellm/pull/29542)
- fix: missing mcp otel attributes by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29554](https://github.com/BerriAI/litellm/pull/29554)
- fix(proxy): resolve managed video model ids for auth by [@&#8203;shivamrawat1](https://github.com/shivamrawat1) in [#&#8203;29545](https://github.com/BerriAI/litellm/pull/29545)
- fix(key\_generate): allow team members to create keys on org-scoped teams by [@&#8203;milan-berri](https://github.com/milan-berri) in [#&#8203;29310](https://github.com/BerriAI/litellm/pull/29310)
- test(pass-through): move Gemini pass-through tests to gemini-3.1-flash-lite by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29595](https://github.com/BerriAI/litellm/pull/29595)
- Litellm oss staging 030626 by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29578](https://github.com/BerriAI/litellm/pull/29578)
- Fix : a2a bugs 030626 by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29566](https://github.com/BerriAI/litellm/pull/29566)
- \[internal copy of [#&#8203;29533](https://github.com/BerriAI/litellm/issues/29533)] fix(anthropic/adapter): emit thinking block for reasoning\_content-only streaming chunks by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29600](https://github.com/BerriAI/litellm/pull/29600)
- ci: reproduce default-Windows wheel install to guard MAX\_PATH by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29597](https://github.com/BerriAI/litellm/pull/29597)
- fix(vertex): strip output\_config.effort for Vertex Claude models that reject it (Haiku 4.5) by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29585](https://github.com/BerriAI/litellm/pull/29585)
- Litellm websocket improvements by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29563](https://github.com/BerriAI/litellm/pull/29563)
- feat(arize/phoenix): OpenInference rendering parity — tool\_calls, cost, passthrough I/O, session/user, multimodal, cache tokens by [@&#8203;milan-berri](https://github.com/milan-berri) in [#&#8203;28800](https://github.com/BerriAI/litellm/pull/28800)
- \[internal copy of [#&#8203;29550](https://github.com/BerriAI/litellm/issues/29550)] fix: passthrough endpoints duplicate logs by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29598](https://github.com/BerriAI/litellm/pull/29598)
- fix(ci): keep coverage rename green when a parallel node runs no tests by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29608](https://github.com/BerriAI/litellm/pull/29608)
- test(vcr): close out the remaining VCR live-call leaks by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29603](https://github.com/BerriAI/litellm/pull/29603)
- fix(key\_generate): exempt UI/CLI session tokens from the budget ceiling for team keys by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29612](https://github.com/BerriAI/litellm/pull/29612)
- fix(realtime): allow null transcripts in stream logging payloads by [@&#8203;milan-berri](https://github.com/milan-berri) in [#&#8203;29625](https://github.com/BerriAI/litellm/pull/29625)
- build(ui): migrate eslint to flat config + bump eslint-config-next to 16 by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29626](https://github.com/BerriAI/litellm/pull/29626)
- fix(key\_generate): scope session-token team-key budget exemption to caller-supplied team\_id by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29641](https://github.com/BerriAI/litellm/pull/29641)
- fix(proxy): disable proxy buffering on streaming SSE responses by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29557](https://github.com/BerriAI/litellm/pull/29557)
- fix(mcp): gate /public/mcp\_hub strictly on litellm.public\_mcp\_servers by [@&#8203;michelligabriele](https://github.com/michelligabriele) in [#&#8203;27764](https://github.com/BerriAI/litellm/pull/27764)
- ci(ui): frontend-lint job enforcing prettier + eslint on changed files by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29633](https://github.com/BerriAI/litellm/pull/29633)
- fix(gemini): googleSearch + server-side tools and googleMaps JSON schema by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29582](https://github.com/BerriAI/litellm/pull/29582)
- fix(proxy): passthrough 404 when SERVER\_ROOT\_PATH is set by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29658](https://github.com/BerriAI/litellm/pull/29658)
- fix(gemini-realtime): use GA event names for Pipecat 1.3.x compatibility by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29662](https://github.com/BerriAI/litellm/pull/29662)
- Litellm oss staging 040626 by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29671](https://github.com/BerriAI/litellm/pull/29671)
- style(ui): prettier formatting pass over the dashboard by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29622](https://github.com/BerriAI/litellm/pull/29622)
- chore: ignore prettier dashboard reformat in git blame by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29695](https://github.com/BerriAI/litellm/pull/29695)
- fix(helm): Enable Backend Deployment to mount Gateway config.yaml by [@&#8203;tin-berri](https://github.com/tin-berri) in [#&#8203;29605](https://github.com/BerriAI/litellm/pull/29605)
- \[internal copy of [#&#8203;29277](https://github.com/BerriAI/litellm/issues/29277)] fix(proxy): add default=None to LiteLLM\_TeamMembership.litellm\_budget\_table by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29684](https://github.com/BerriAI/litellm/pull/29684)
- test: make custom\_tokenizer proxy tests hermetic by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29643](https://github.com/BerriAI/litellm/pull/29643)
- test(proxy): stop running real-DB tests in GitHub Actions unit jobs by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29700](https://github.com/BerriAI/litellm/pull/29700)
- chore(ui): remove the bare-fetch lint rule by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29712](https://github.com/BerriAI/litellm/pull/29712)
- Litellm jwt mapping virtualkeys by [@&#8203;shivamrawat1](https://github.com/shivamrawat1) in [#&#8203;28510](https://github.com/BerriAI/litellm/pull/28510)
- refactor(ui): shared HTTP client + location-pinned fetch() lint rule by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29723](https://github.com/BerriAI/litellm/pull/29723)
- fix(proxy): stop team BYOK model name corruption on model edit by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29731](https://github.com/BerriAI/litellm/pull/29731)
- \[internal copy of [#&#8203;29511](https://github.com/BerriAI/litellm/issues/29511)] feat(guardrails): add sensitive data routing to on-premise models by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29531](https://github.com/BerriAI/litellm/pull/29531)
- fix(proxy/hooks): populate llm\_provider on internal rate-limit errors by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;27707](https://github.com/BerriAI/litellm/pull/27707)
- fix(vertex/anthropic): handle namespace tools and strip client\_metadata for codex compatibility by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29489](https://github.com/BerriAI/litellm/pull/29489)
- Support OAuth M2M for Databricks Apps A2A agents by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29586](https://github.com/BerriAI/litellm/pull/29586)
- fix: small CLAUDE.md nit by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29749](https://github.com/BerriAI/litellm/pull/29749)
- fix(anthropic): route Claude Opus 4.8 through adaptive thinking by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29702](https://github.com/BerriAI/litellm/pull/29702)
- fix(proxy): persist oauth2\_flow on MCP server registration by [@&#8203;michelligabriele](https://github.com/michelligabriele) in [#&#8203;29690](https://github.com/BerriAI/litellm/pull/29690)
- \[internal copy of [#&#8203;27491](https://github.com/BerriAI/litellm/issues/27491)] fix(realtime): Fix Realtime Audio Token Cost Tracking by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29722](https://github.com/BerriAI/litellm/pull/29722)
- fix(galileo): use ingest traces API and standard logging payload by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29651](https://github.com/BerriAI/litellm/pull/29651)
- fix(auth): expand all-team-models sentinel in can\_key\_call\_model for batch validation by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29746](https://github.com/BerriAI/litellm/pull/29746)
- test(vcr): stop refreshing cassette TTL on read so cassettes lapse after 24h by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29784](https://github.com/BerriAI/litellm/pull/29784)
- test(ci): record/replay OpenAI image gen so the spend E2E isn't outage-bound by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29787](https://github.com/BerriAI/litellm/pull/29787)
- fix(ui): route MCP playground auth by oauth2 mode instead of token\_url by [@&#8203;tin-berri](https://github.com/tin-berri) in [#&#8203;29714](https://github.com/BerriAI/litellm/pull/29714)
- refactor(ui): centralize proxy base URL resolution into tested resolver by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29793](https://github.com/BerriAI/litellm/pull/29793)
- Litellm oss staging 050626 by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29774](https://github.com/BerriAI/litellm/pull/29774)
- test(google): add google-genai SDK proxy integration tests by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29781](https://github.com/BerriAI/litellm/pull/29781)
- fix(jwt): use resolved DB user\_id for spend on legacy email match by [@&#8203;milan-berri](https://github.com/milan-berri) in [#&#8203;29217](https://github.com/BerriAI/litellm/pull/29217)
- feat(ui): generate dashboard API types from the proxy OpenAPI spec by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29816](https://github.com/BerriAI/litellm/pull/29816)
- fix(proxy): drop deleted team BYOK model name from team.models by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29820](https://github.com/BerriAI/litellm/pull/29820)
- feat(mcp): per-server env vars with global + per-user scopes by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;28917](https://github.com/BerriAI/litellm/pull/28917)
- refactor(ui): route behavior-preserving networking calls through apiClient by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29806](https://github.com/BerriAI/litellm/pull/29806)
- fix(mcp): persist Tools-tab MCP OAuth token to DB by [@&#8203;tin-berri](https://github.com/tin-berri) in [#&#8203;29809](https://github.com/BerriAI/litellm/pull/29809)
- fix(ui): require new expiration when regenerating an expired key by [@&#8203;milan-berri](https://github.com/milan-berri) in [#&#8203;29838](https://github.com/BerriAI/litellm/pull/29838)
- refactor(ui): route query-building networking calls through apiClient by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29815](https://github.com/BerriAI/litellm/pull/29815)
- Make the image-gen record/replay proxy report cache mode and per-request HIT/MISS by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29802](https://github.com/BerriAI/litellm/pull/29802)
- feat(proxy): hot-reload .env in dev when running with --reload by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29783](https://github.com/BerriAI/litellm/pull/29783)
- fix(ui): stop MCP playground tool calls from sending twice by [@&#8203;tin-berri](https://github.com/tin-berri) in [#&#8203;29821](https://github.com/BerriAI/litellm/pull/29821)
- feat(fal\_ai): add Nano Banana / Gemini 2.5 Flash Image generation support by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29798](https://github.com/BerriAI/litellm/pull/29798)
- Title: Fix managed batch cancel credential resolution by [@&#8203;shivamrawat1](https://github.com/shivamrawat1) in [#&#8203;29734](https://github.com/BerriAI/litellm/pull/29734)
- Title: fix(proxy): resolve vector store file list credentials from team deployments by [@&#8203;shivamrawat1](https://github.com/shivamrawat1) in [#&#8203;29739](https://github.com/BerriAI/litellm/pull/29739)
- refactor: convert AWS and GCP Terraform stacks into reusable modules … by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;28103](https://github.com/BerriAI/litellm/pull/28103)
- chore(ui): build ui for release by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29853](https://github.com/BerriAI/litellm/pull/29853)
- fix(terraform/gcp): prompt for image\_registry in DeployStack one-click by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29852](https://github.com/BerriAI/litellm/pull/29852)
- fix(terraform/gcp): abandon SQL user on destroy by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29855](https://github.com/BerriAI/litellm/pull/29855)
- Extend the record/replay proxy to chat, embeddings, moderations, rerank, and Anthropic by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29847](https://github.com/BerriAI/litellm/pull/29847)
- chore(deps): bump deps by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29860](https://github.com/BerriAI/litellm/pull/29860)
- chore(ci): promote internal staging to main by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29861](https://github.com/BerriAI/litellm/pull/29861)
- fix: 400 on Anthropic context overflow; seed identity on failed auth by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29848](https://github.com/BerriAI/litellm/pull/29848)
- chore(ci): promote internal staging to main by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29862](https://github.com/BerriAI/litellm/pull/29862)
- chore(release): patch v1.89.0-rc.1 with [#&#8203;30064](https://github.com/BerriAI/litellm/issues/30064) (Claude Fable 5) for v1.89.0-rc.2 by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;30143](https://github.com/BerriAI/litellm/pull/30143)

**Full Changelog**: <https://github.com/BerriAI/litellm/compare/v1.88.0...v1.89.0>

### [`v1.89.0`](https://github.com/BerriAI/litellm/releases/tag/v1.89.0)

[Compare Source](https://github.com/BerriAI/litellm/compare/v1.88.2...v1.89.0)

##### Verify Docker Image Signature

All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0).

**Verify using the pinned commit hash (recommended):**

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

```bash
cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
  ghcr.io/berriai/litellm:v1.89.0
```

**Verify using the release tag (convenience):**

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

```bash
cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/v1.89.0/cosign.pub \
  ghcr.io/berriai/litellm:v1.89.0
```

Expected output:

```
The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key
```

***

##### What's Changed

- test(responses): bump deprecated gemini-3-pro-preview to gemini-3.1-pro-preview by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29433](https://github.com/BerriAI/litellm/pull/29433)
- fix: map mistral/ministral-8b-latest in model price map by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29453](https://github.com/BerriAI/litellm/pull/29453)
- fix(datadog): split oversized batches on 413 instead of re-queueing forever by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29444](https://github.com/BerriAI/litellm/pull/29444)
- feat(otel): allowlist team\_metadata sub-keys promoted to baggage by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29442](https://github.com/BerriAI/litellm/pull/29442)
- fix: stop use\_chat\_completions\_api flag from leaking into provider request body by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29447](https://github.com/BerriAI/litellm/pull/29447)
- fix(anthropic, fireworks): inline legacy $ref defs in tool schemas by [@&#8203;milan-berri](https://github.com/milan-berri) in [#&#8203;28646](https://github.com/BerriAI/litellm/pull/28646)
- fix(proxy): omit OpenAI \[DONE] on google-genai streamGenerateContent by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29426](https://github.com/BerriAI/litellm/pull/29426)
- ci(release): create stable/X.Y.x line branch on X.Y.0 tags by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29457](https://github.com/BerriAI/litellm/pull/29457)
- fix(vector-stores): support engines URL for Vertex AI Search by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;27885](https://github.com/BerriAI/litellm/pull/27885)
- fix(ui): render caller-supplied filter options in caller order by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29462](https://github.com/BerriAI/litellm/pull/29462)
- fix(batches): skip unnecessary batch input file reads by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29114](https://github.com/BerriAI/litellm/pull/29114)
- docs(agents): clarify when to create new test files by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29472](https://github.com/BerriAI/litellm/pull/29472)
- Litellm OSS Staging by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29161](https://github.com/BerriAI/litellm/pull/29161)
- fix(mcp): clear allowed\_tools and tool overrides on MCP server edit by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29411](https://github.com/BerriAI/litellm/pull/29411)
- Litellm OSS Staging 010626 by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29422](https://github.com/BerriAI/litellm/pull/29422)
- fix(ci): make CircleCI rerun-failed-tests collect tests when 2+ test files fail by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29475](https://github.com/BerriAI/litellm/pull/29475)
- feat(a2a): watsonx Orchestrate agent provider by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29410](https://github.com/BerriAI/litellm/pull/29410)
- fix(azure\_ai): strip tool-level extra fields on 400 and retry by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29479](https://github.com/BerriAI/litellm/pull/29479)
- fix(docs): remove fixed dimensions from README hero image by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29496](https://github.com/BerriAI/litellm/pull/29496)
- Litellm oss staging by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29492](https://github.com/BerriAI/litellm/pull/29492)
- fix: small CLAUDE.md nits by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29504](https://github.com/BerriAI/litellm/pull/29504)
- Add MCP semantic conventions to otelv2 by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29468](https://github.com/BerriAI/litellm/pull/29468)
- fix(passthrough): emit otel guardrail span when a guardrail blocks by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29470](https://github.com/BerriAI/litellm/pull/29470)
- fix(proxy): strip NUL bytes from spend log payloads to prevent PostgreSQL 22P05 by [@&#8203;milan-berri](https://github.com/milan-berri) in [#&#8203;29515](https://github.com/BerriAI/litellm/pull/29515)
- \[internal copy of [#&#8203;28008](https://github.com/BerriAI/litellm/issues/28008)] Support MCP OAuth passthrough and issuer-scoped JWT auth by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;28356](https://github.com/BerriAI/litellm/pull/28356)
- feat(vector-stores): forward per-request params to Vertex AI Search by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29459](https://github.com/BerriAI/litellm/pull/29459)
- feat(proxy): add per-MCP-server RPM rate limiting for keys and teams by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29482](https://github.com/BerriAI/litellm/pull/29482)
- fix(tests): drop module-level test calls that break local\_testing collection by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29520](https://github.com/BerriAI/litellm/pull/29520)
- feat(agents): add LangFlow agent provider with A2A session bridging by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28963](https://github.com/BerriAI/litellm/pull/28963)
- fix(ui/agents): make A2A skill tags enterable and validated by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29512](https://github.com/BerriAI/litellm/pull/29512)
- \[internal copy of [#&#8203;29232](https://github.com/BerriAI/litellm/issues/29232)] feat: route future Claude models to Anthropic provider via pattern matching by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29239](https://github.com/BerriAI/litellm/pull/29239)
- fix(tests): drop import-time completion call in test\_register\_model by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29521](https://github.com/BerriAI/litellm/pull/29521)
- test: stabilize batch VCR coverage and stop live upload/network leaks by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29477](https://github.com/BerriAI/litellm/pull/29477)
- \[internal copy of [#&#8203;29003](https://github.com/BerriAI/litellm/issues/29003)] fix(vertex\_ai): use user-supplied api\_base as is for Model Garden OpenAI-compat path by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29530](https://github.com/BerriAI/litellm/pull/29530)
- feat(proxy): native /health/drain preStop hook for graceful shutdown by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29439](https://github.com/BerriAI/litellm/pull/29439)
- fix(auth): preserve 401 status for expired JWTs in OTel traces by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29510](https://github.com/BerriAI/litellm/pull/29510)
- fix(otel): capture 401 error details in management endpoint spans by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29535](https://github.com/BerriAI/litellm/pull/29535)
- test(proxy/utils): pin bottom-of-file helper behavior by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29509](https://github.com/BerriAI/litellm/pull/29509)
- test(proxy/utils): pin PrismaClient and spend-update behavior by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29488](https://github.com/BerriAI/litellm/pull/29488)
- test(proxy/utils): pin ProxyLogging behavior by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29485](https://github.com/BerriAI/litellm/pull/29485)
- fix: missing span for guardrail passthrough by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29552](https://github.com/BerriAI/litellm/pull/29552)
- fix(auth): let internal users view search tools by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29542](https://github.com/BerriAI/litellm/pull/29542)
- fix: missing mcp otel attributes by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29554](https://github.com/BerriAI/litellm/pull/29554)
- fix(proxy): resolve managed video model ids for auth by [@&#8203;shivamrawat1](https://github.com/shivamrawat1) in [#&#8203;29545](https://github.com/BerriAI/litellm/pull/29545)
- fix(key\_generate): allow team members to create keys on org-scoped teams by [@&#8203;milan-berri](https://github.com/milan-berri) in [#&#8203;29310](https://github.com/BerriAI/litellm/pull/29310)
- test(pass-through): move Gemini pass-through tests to gemini-3.1-flash-lite by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29595](https://github.com/BerriAI/litellm/pull/29595)
- Litellm oss staging 030626 by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29578](https://github.com/BerriAI/litellm/pull/29578)
- Fix : a2a bugs 030626 by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29566](https://github.com/BerriAI/litellm/pull/29566)
- \[internal copy of [#&#8203;29533](https://github.com/BerriAI/litellm/issues/29533)] fix(anthropic/adapter): emit thinking block for reasoning\_content-only streaming chunks by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29600](https://github.com/BerriAI/litellm/pull/29600)
- ci: reproduce default-Windows wheel install to guard MAX\_PATH by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29597](https://github.com/BerriAI/litellm/pull/29597)
- fix(vertex): strip output\_config.effort for Vertex Claude models that reject it (Haiku 4.5) by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29585](https://github.com/BerriAI/litellm/pull/29585)
- Litellm websocket improvements by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29563](https://github.com/BerriAI/litellm/pull/29563)
- feat(arize/phoenix): OpenInference rendering parity — tool\_calls, cost, passthrough I/O, session/user, multimodal, cache tokens by [@&#8203;milan-berri](https://github.com/milan-berri) in [#&#8203;28800](https://github.com/BerriAI/litellm/pull/28800)
- \[internal copy of [#&#8203;29550](https://github.com/BerriAI/litellm/issues/29550)] fix: passthrough endpoints duplicate logs by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29598](https://github.com/BerriAI/litellm/pull/29598)
- fix(ci): keep coverage rename green when a parallel node runs no tests by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29608](https://github.com/BerriAI/litellm/pull/29608)
- test(vcr): close out the remaining VCR live-call leaks by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29603](https://github.com/BerriAI/litellm/pull/29603)
- fix(key\_generate): exempt UI/CLI session tokens from the budget ceiling for team keys by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29612](https://github.com/BerriAI/litellm/pull/29612)
- fix(realtime): allow null transcripts in stream logging payloads by [@&#8203;milan-berri](https://github.com/milan-berri) in [#&#8203;29625](https://github.com/BerriAI/litellm/pull/29625)
- build(ui): migrate eslint to flat config + bump eslint-config-next to 16 by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29626](https://github.com/BerriAI/litellm/pull/29626)
- fix(key\_generate): scope session-token team-key budget exemption to caller-supplied team\_id by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29641](https://github.com/BerriAI/litellm/pull/29641)
- fix(proxy): disable proxy buffering on streaming SSE responses by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29557](https://github.com/BerriAI/litellm/pull/29557)
- fix(mcp): gate /public/mcp\_hub strictly on litellm.public\_mcp\_servers by [@&#8203;michelligabriele](https://github.com/michelligabriele) in [#&#8203;27764](https://github.com/BerriAI/litellm/pull/27764)
- ci(ui): frontend-lint job enforcing prettier + eslint on changed files by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29633](https://github.com/BerriAI/litellm/pull/29633)
- fix(gemini): googleSearch + server-side tools and googleMaps JSON schema by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29582](https://github.com/BerriAI/litellm/pull/29582)
- fix(proxy): passthrough 404 when SERVER\_ROOT\_PATH is set by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29658](https://github.com/BerriAI/litellm/pull/29658)
- fix(gemini-realtime): use GA event names for Pipecat 1.3.x compatibility by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29662](https://github.com/BerriAI/litellm/pull/29662)
- Litellm oss staging 040626 by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29671](https://github.com/BerriAI/litellm/pull/29671)
- style(ui): prettier formatting pass over the dashboard by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29622](https://github.com/BerriAI/litellm/pull/29622)
- chore: ignore prettier dashboard reformat in git blame by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29695](https://github.com/BerriAI/litellm/pull/29695)
- fix(helm): Enable Backend Deployment to mount Gateway config.yaml by [@&#8203;tin-berri](https://github.com/tin-berri) in [#&#8203;29605](https://github.com/BerriAI/litellm/pull/29605)
- \[internal copy of [#&#8203;29277](https://github.com/BerriAI/litellm/issues/29277)] fix(proxy): add default=None to LiteLLM\_TeamMembership.litellm\_budget\_table by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29684](https://github.com/BerriAI/litellm/pull/29684)
- test: make custom\_tokenizer proxy tests hermetic by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29643](https://github.com/BerriAI/litellm/pull/29643)
- test(proxy): stop running real-DB tests in GitHub Actions unit jobs by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29700](https://github.com/BerriAI/litellm/pull/29700)
- chore(ui): remove the bare-fetch lint rule by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29712](https://github.com/BerriAI/litellm/pull/29712)
- Litellm jwt mapping virtualkeys by [@&#8203;shivamrawat1](https://github.com/shivamrawat1) in [#&#8203;28510](https://github.com/BerriAI/litellm/pull/28510)
- refactor(ui): shared HTTP client + location-pinned fetch() lint rule by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29723](https://github.com/BerriAI/litellm/pull/29723)
- fix(proxy): stop team BYOK model name corruption on model edit by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29731](https://github.com/BerriAI/litellm/pull/29731)
- \[internal copy of [#&#8203;29511](https://github.com/BerriAI/litellm/issues/29511)] feat(guardrails): add sensitive data routing to on-premise models by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29531](https://github.com/BerriAI/litellm/pull/29531)
- fix(proxy/hooks): populate llm\_provider on internal rate-limit errors by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;27707](https://github.com/BerriAI/litellm/pull/27707)
- fix(vertex/anthropic): handle namespace tools and strip client\_metadata for codex compatibility by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29489](https://github.com/BerriAI/litellm/pull/29489)
- Support OAuth M2M for Databricks Apps A2A agents by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29586](https://github.com/BerriAI/litellm/pull/29586)
- fix: small CLAUDE.md nit by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29749](https://github.com/BerriAI/litellm/pull/29749)
- fix(anthropic): route Claude Opus 4.8 through adaptive thinking by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29702](https://github.com/BerriAI/litellm/pull/29702)
- fix(proxy): persist oauth2\_flow on MCP server registration by [@&#8203;michelligabriele](https://github.com/michelligabriele) in [#&#8203;29690](https://github.com/BerriAI/litellm/pull/29690)
- \[internal copy of [#&#8203;27491](https://github.com/BerriAI/litellm/issues/27491)] fix(realtime): Fix Realtime Audio Token Cost Tracking by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29722](https://github.com/BerriAI/litellm/pull/29722)
- fix(galileo): use ingest traces API and standard logging payload by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29651](https://github.com/BerriAI/litellm/pull/29651)
- fix(auth): expand all-team-models sentinel in can\_key\_call\_model for batch validation by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29746](https://github.com/BerriAI/litellm/pull/29746)
- test(vcr): stop refreshing cassette TTL on read so cassettes lapse after 24h by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29784](https://github.com/BerriAI/litellm/pull/29784)
- test(ci): record/replay OpenAI image gen so the spend E2E isn't outage-bound by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29787](https://github.com/BerriAI/litellm/pull/29787)
- fix(ui): route MCP playground auth by oauth2 mode instead of token\_url by [@&#8203;tin-berri](https://github.com/tin-berri) in [#&#8203;29714](https://github.com/BerriAI/litellm/pull/29714)
- refactor(ui): centralize proxy base URL resolution into tested resolver by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29793](https://github.com/BerriAI/litellm/pull/29793)
- Litellm oss staging 050626 by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29774](https://github.com/BerriAI/litellm/pull/29774)
- test(google): add google-genai SDK proxy integration tests by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29781](https://github.com/BerriAI/litellm/pull/29781)
- fix(jwt): use resolved DB user\_id for spend on legacy email match by [@&#8203;milan-berri](https://github.com/milan-berri) in [#&#8203;29217](https://github.com/BerriAI/litellm/pull/29217)
- feat(ui): generate dashboard API types from the proxy OpenAPI spec by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29816](https://github.com/BerriAI/litellm/pull/29816)
- fix(proxy): drop deleted team BYOK model name from team.models by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29820](https://github.com/BerriAI/litellm/pull/29820)
- feat(mcp): per-server env vars with global + per-user scopes by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;28917](https://github.com/BerriAI/litellm/pull/28917)
- refactor(ui): route behavior-preserving networking calls through apiClient by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29806](https://github.com/BerriAI/litellm/pull/29806)
- fix(mcp): persist Tools-tab MCP OAuth token to DB by [@&#8203;tin-berri](https://github.com/tin-berri) in [#&#8203;29809](https://github.com/BerriAI/litellm/pull/29809)
- fix(ui): require new expiration when regenerating an expired key by [@&#8203;milan-berri](https://github.com/milan-berri) in [#&#8203;29838](https://github.com/BerriAI/litellm/pull/29838)
- refactor(ui): route query-building networking calls through apiClient by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29815](https://github.com/BerriAI/litellm/pull/29815)
- Make the image-gen record/replay proxy report cache mode and per-request HIT/MISS by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29802](https://github.com/BerriAI/litellm/pull/29802)
- feat(proxy): hot-reload .env in dev when running with --reload by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29783](https://github.com/BerriAI/litellm/pull/29783)
- fix(ui): stop MCP playground tool calls from sending twice by [@&#8203;tin-berri](https://github.com/tin-berri) in [#&#8203;29821](https://github.com/BerriAI/litellm/pull/29821)
- feat(fal\_ai): add Nano Banana / Gemini 2.5 Flash Image generation support by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29798](https://github.com/BerriAI/litellm/pull/29798)
- Title: Fix managed batch cancel credential resolution by [@&#8203;shivamrawat1](https://github.com/shivamrawat1) in [#&#8203;29734](https://github.com/BerriAI/litellm/pull/29734)
- Title: fix(proxy): resolve vector store file list credentials from team deployments by [@&#8203;shivamrawat1](https://github.com/shivamrawat1) in [#&#8203;29739](https://github.com/BerriAI/litellm/pull/29739)
- refactor: convert AWS and GCP Terraform stacks into reusable modules … by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;28103](https://github.com/BerriAI/litellm/pull/28103)
- chore(ui): build ui for release by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29853](https://github.com/BerriAI/litellm/pull/29853)
- fix(terraform/gcp): prompt for image\_registry in DeployStack one-click by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29852](https://github.com/BerriAI/litellm/pull/29852)
- fix(terraform/gcp): abandon SQL user on destroy by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29855](https://github.com/BerriAI/litellm/pull/29855)
- Extend the record/replay proxy to chat, embeddings, moderations, rerank, and Anthropic by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29847](https://github.com/BerriAI/litellm/pull/29847)
- chore(deps): bump deps by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29860](https://github.com/BerriAI/litellm/pull/29860)
- chore(ci): promote internal staging to main by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29861](https://github.com/BerriAI/litellm/pull/29861)
- fix: 400 on Anthropic context overflow; seed identity on failed auth by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29848](https://github.com/BerriAI/litellm/pull/29848)
- chore(ci): promote internal staging to main by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29862](https://github.com/BerriAI/litellm/pull/29862)
- chore(release): patch v1.89.0-rc.1 with [#&#8203;30064](https://github.com/BerriAI/litellm/issues/30064) (Claude Fable 5) for v1.89.0-rc.2 by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;30143](https://github.com/BerriAI/litellm/pull/30143)

**Full Changelog**: <https://github.com/BerriAI/litellm/compare/v1.88.0...v1.89.0>

### [`v1.88.2`](https://github.com/BerriAI/litellm/releases/tag/v1.88.2)

[Compare Source](https://github.com/BerriAI/litellm/compare/v1.88.2...v1.88.2)

##### Verify Docker Image Signature

All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0).

**Verify using the pinned commit hash (recommended):**

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

```bash
cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
  ghcr.io/berriai/litellm:v1.88.2
```

**Verify using the release tag (convenience):**

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

```bash
cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/v1.88.2/cosign.pub \
  ghcr.io/berriai/litellm:v1.88.2
```

Expected output:

```
The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key
```

***

##### What's Changed

- chore(release): backport Fable 5, batch-file auth, CrowdStrike AIDR, Mantle Responses SigV4, and NetApp streaming-cost fix to stable/1.88.x and cut 1.88.2 by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;30144](https://github.com/BerriAI/litellm/pull/30144)
- chore(release): backport DB-resilience, passthrough, model-info, budget, and deps fixes to stable/1.88.x by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;30408](https://github.com/BerriAI/litellm/pull/30408)

**Full Changelog**: <https://github.com/BerriAI/litellm/compare/v1.88.1...v1.88.2>

### [`v1.88.2`](https://github.com/BerriAI/litellm/releases/tag/v1.88.2)

[Compare Source](https://github.com/BerriAI/litellm/compare/v1.88.1...v1.88.2)

##### Verify Docker Image Signature

All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0).

**Verify using the pinned commit hash (recommended):**

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

```bash
cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
  ghcr.io/berriai/litellm:v1.88.2
```

**Verify using the release tag (convenience):**

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

```bash
cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/v1.88.2/cosign.pub \
  ghcr.io/berriai/litellm:v1.88.2
```

Expected output:

```
The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key
```

***

##### What's Changed

- chore(release): backport Fable 5, batch-file auth, CrowdStrike AIDR, Mantle Responses SigV4, and NetApp streaming-cost fix to stable/1.88.x and cut 1.88.2 by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;30144](https://github.com/BerriAI/litellm/pull/30144)
- chore(release): backport DB-resilience, passthrough, model-info, budget, and deps fixes to stable/1.88.x by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;30408](https://github.com/BerriAI/litellm/pull/30408)

**Full Changelog**: <https://github.com/BerriAI/litellm/compare/v1.88.1...v1.88.2>

</details>

---

### Configuration

📅 **Schedule**: (in timezone Europe/London)

- Branch creation
  - At any time (no schedule defined)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about these updates again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4yMTkuMCIsInVwZGF0ZWRJblZlciI6IjQzLjIxOS4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJyZW5vdmF0ZS9jb250YWluZXIiLCJ0eXBlL21pbm9yIl19-->

Reviewed-on: https://forgejo.hayden.moe/hayden/phoebe/pulls/93
blake-hamm added a commit to blake-hamm/bhamm-lab that referenced this pull request Jun 16, 2026
…to v1.89.0 (#200)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [https://github.com/BerriAI/litellm.git](https://github.com/BerriAI/litellm) | minor | `v1.85.1` → `v1.89.0` |

---

> ⚠️ **Warning**
>
> Some dependencies could not be looked up. Check the [Dependency Dashboard](issues/155) for more information.

---

### Release Notes

<details>
<summary>BerriAI/litellm (https://github.com/BerriAI/litellm.git)</summary>

### [`v1.89.0`](https://github.com/BerriAI/litellm/releases/tag/v1.89.0)

[Compare Source](https://github.com/BerriAI/litellm/compare/v1.88.2...v1.89.0)

#### Verify Docker Image Signature

All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0).

**Verify using the pinned commit hash (recommended):**

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

```bash
cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
  ghcr.io/berriai/litellm:v1.89.0
```

**Verify using the release tag (convenience):**

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

```bash
cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/v1.89.0/cosign.pub \
  ghcr.io/berriai/litellm:v1.89.0
```

Expected output:

```
The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key
```

***

#### What's Changed

- test(responses): bump deprecated gemini-3-pro-preview to gemini-3.1-pro-preview by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29433](https://github.com/BerriAI/litellm/pull/29433)
- fix: map mistral/ministral-8b-latest in model price map by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29453](https://github.com/BerriAI/litellm/pull/29453)
- fix(datadog): split oversized batches on 413 instead of re-queueing forever by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29444](https://github.com/BerriAI/litellm/pull/29444)
- feat(otel): allowlist team\_metadata sub-keys promoted to baggage by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29442](https://github.com/BerriAI/litellm/pull/29442)
- fix: stop use\_chat\_completions\_api flag from leaking into provider request body by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29447](https://github.com/BerriAI/litellm/pull/29447)
- fix(anthropic, fireworks): inline legacy $ref defs in tool schemas by [@&#8203;milan-berri](https://github.com/milan-berri) in [#&#8203;28646](https://github.com/BerriAI/litellm/pull/28646)
- fix(proxy): omit OpenAI \[DONE] on google-genai streamGenerateContent by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29426](https://github.com/BerriAI/litellm/pull/29426)
- ci(release): create stable/X.Y.x line branch on X.Y.0 tags by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29457](https://github.com/BerriAI/litellm/pull/29457)
- fix(vector-stores): support engines URL for Vertex AI Search by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;27885](https://github.com/BerriAI/litellm/pull/27885)
- fix(ui): render caller-supplied filter options in caller order by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29462](https://github.com/BerriAI/litellm/pull/29462)
- fix(batches): skip unnecessary batch input file reads by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29114](https://github.com/BerriAI/litellm/pull/29114)
- docs(agents): clarify when to create new test files by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29472](https://github.com/BerriAI/litellm/pull/29472)
- Litellm OSS Staging by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29161](https://github.com/BerriAI/litellm/pull/29161)
- fix(mcp): clear allowed\_tools and tool overrides on MCP server edit by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29411](https://github.com/BerriAI/litellm/pull/29411)
- Litellm OSS Staging 010626 by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29422](https://github.com/BerriAI/litellm/pull/29422)
- fix(ci): make CircleCI rerun-failed-tests collect tests when 2+ test files fail by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29475](https://github.com/BerriAI/litellm/pull/29475)
- feat(a2a): watsonx Orchestrate agent provider by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29410](https://github.com/BerriAI/litellm/pull/29410)
- fix(azure\_ai): strip tool-level extra fields on 400 and retry by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29479](https://github.com/BerriAI/litellm/pull/29479)
- fix(docs): remove fixed dimensions from README hero image by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29496](https://github.com/BerriAI/litellm/pull/29496)
- Litellm oss staging by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29492](https://github.com/BerriAI/litellm/pull/29492)
- fix: small CLAUDE.md nits by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29504](https://github.com/BerriAI/litellm/pull/29504)
- Add MCP semantic conventions to otelv2 by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29468](https://github.com/BerriAI/litellm/pull/29468)
- fix(passthrough): emit otel guardrail span when a guardrail blocks by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29470](https://github.com/BerriAI/litellm/pull/29470)
- fix(proxy): strip NUL bytes from spend log payloads to prevent PostgreSQL 22P05 by [@&#8203;milan-berri](https://github.com/milan-berri) in [#&#8203;29515](https://github.com/BerriAI/litellm/pull/29515)
- \[internal copy of [#&#8203;28008](https://github.com/BerriAI/litellm/issues/28008)] Support MCP OAuth passthrough and issuer-scoped JWT auth by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;28356](https://github.com/BerriAI/litellm/pull/28356)
- feat(vector-stores): forward per-request params to Vertex AI Search by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29459](https://github.com/BerriAI/litellm/pull/29459)
- feat(proxy): add per-MCP-server RPM rate limiting for keys and teams by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29482](https://github.com/BerriAI/litellm/pull/29482)
- fix(tests): drop module-level test calls that break local\_testing collection by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29520](https://github.com/BerriAI/litellm/pull/29520)
- feat(agents): add LangFlow agent provider with A2A session bridging by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28963](https://github.com/BerriAI/litellm/pull/28963)
- fix(ui/agents): make A2A skill tags enterable and validated by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29512](https://github.com/BerriAI/litellm/pull/29512)
- \[internal copy of [#&#8203;29232](https://github.com/BerriAI/litellm/issues/29232)] feat: route future Claude models to Anthropic provider via pattern matching by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29239](https://github.com/BerriAI/litellm/pull/29239)
- fix(tests): drop import-time completion call in test\_register\_model by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29521](https://github.com/BerriAI/litellm/pull/29521)
- test: stabilize batch VCR coverage and stop live upload/network leaks by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29477](https://github.com/BerriAI/litellm/pull/29477)
- \[internal copy of [#&#8203;29003](https://github.com/BerriAI/litellm/issues/29003)] fix(vertex\_ai): use user-supplied api\_base as is for Model Garden OpenAI-compat path by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29530](https://github.com/BerriAI/litellm/pull/29530)
- feat(proxy): native /health/drain preStop hook for graceful shutdown by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29439](https://github.com/BerriAI/litellm/pull/29439)
- fix(auth): preserve 401 status for expired JWTs in OTel traces by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29510](https://github.com/BerriAI/litellm/pull/29510)
- fix(otel): capture 401 error details in management endpoint spans by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29535](https://github.com/BerriAI/litellm/pull/29535)
- test(proxy/utils): pin bottom-of-file helper behavior by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29509](https://github.com/BerriAI/litellm/pull/29509)
- test(proxy/utils): pin PrismaClient and spend-update behavior by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29488](https://github.com/BerriAI/litellm/pull/29488)
- test(proxy/utils): pin ProxyLogging behavior by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29485](https://github.com/BerriAI/litellm/pull/29485)
- fix: missing span for guardrail passthrough by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29552](https://github.com/BerriAI/litellm/pull/29552)
- fix(auth): let internal users view search tools by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29542](https://github.com/BerriAI/litellm/pull/29542)
- fix: missing mcp otel attributes by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29554](https://github.com/BerriAI/litellm/pull/29554)
- fix(proxy): resolve managed video model ids for auth by [@&#8203;shivamrawat1](https://github.com/shivamrawat1) in [#&#8203;29545](https://github.com/BerriAI/litellm/pull/29545)
- fix(key\_generate): allow team members to create keys on org-scoped teams by [@&#8203;milan-berri](https://github.com/milan-berri) in [#&#8203;29310](https://github.com/BerriAI/litellm/pull/29310)
- test(pass-through): move Gemini pass-through tests to gemini-3.1-flash-lite by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29595](https://github.com/BerriAI/litellm/pull/29595)
- Litellm oss staging 030626 by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29578](https://github.com/BerriAI/litellm/pull/29578)
- Fix : a2a bugs 030626 by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29566](https://github.com/BerriAI/litellm/pull/29566)
- \[internal copy of [#&#8203;29533](https://github.com/BerriAI/litellm/issues/29533)] fix(anthropic/adapter): emit thinking block for reasoning\_content-only streaming chunks by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29600](https://github.com/BerriAI/litellm/pull/29600)
- ci: reproduce default-Windows wheel install to guard MAX\_PATH by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29597](https://github.com/BerriAI/litellm/pull/29597)
- fix(vertex): strip output\_config.effort for Vertex Claude models that reject it (Haiku 4.5) by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29585](https://github.com/BerriAI/litellm/pull/29585)
- Litellm websocket improvements by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29563](https://github.com/BerriAI/litellm/pull/29563)
- feat(arize/phoenix): OpenInference rendering parity — tool\_calls, cost, passthrough I/O, session/user, multimodal, cache tokens by [@&#8203;milan-berri](https://github.com/milan-berri) in [#&#8203;28800](https://github.com/BerriAI/litellm/pull/28800)
- \[internal copy of [#&#8203;29550](https://github.com/BerriAI/litellm/issues/29550)] fix: passthrough endpoints duplicate logs by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29598](https://github.com/BerriAI/litellm/pull/29598)
- fix(ci): keep coverage rename green when a parallel node runs no tests by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29608](https://github.com/BerriAI/litellm/pull/29608)
- test(vcr): close out the remaining VCR live-call leaks by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29603](https://github.com/BerriAI/litellm/pull/29603)
- fix(key\_generate): exempt UI/CLI session tokens from the budget ceiling for team keys by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29612](https://github.com/BerriAI/litellm/pull/29612)
- fix(realtime): allow null transcripts in stream logging payloads by [@&#8203;milan-berri](https://github.com/milan-berri) in [#&#8203;29625](https://github.com/BerriAI/litellm/pull/29625)
- build(ui): migrate eslint to flat config + bump eslint-config-next to 16 by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29626](https://github.com/BerriAI/litellm/pull/29626)
- fix(key\_generate): scope session-token team-key budget exemption to caller-supplied team\_id by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29641](https://github.com/BerriAI/litellm/pull/29641)
- fix(proxy): disable proxy buffering on streaming SSE responses by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29557](https://github.com/BerriAI/litellm/pull/29557)
- fix(mcp): gate /public/mcp\_hub strictly on litellm.public\_mcp\_servers by [@&#8203;michelligabriele](https://github.com/michelligabriele) in [#&#8203;27764](https://github.com/BerriAI/litellm/pull/27764)
- ci(ui): frontend-lint job enforcing prettier + eslint on changed files by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29633](https://github.com/BerriAI/litellm/pull/29633)
- fix(gemini): googleSearch + server-side tools and googleMaps JSON schema by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29582](https://github.com/BerriAI/litellm/pull/29582)
- fix(proxy): passthrough 404 when SERVER\_ROOT\_PATH is set by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29658](https://github.com/BerriAI/litellm/pull/29658)
- fix(gemini-realtime): use GA event names for Pipecat 1.3.x compatibility by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29662](https://github.com/BerriAI/litellm/pull/29662)
- Litellm oss staging 040626 by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29671](https://github.com/BerriAI/litellm/pull/29671)
- style(ui): prettier formatting pass over the dashboard by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29622](https://github.com/BerriAI/litellm/pull/29622)
- chore: ignore prettier dashboard reformat in git blame by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29695](https://github.com/BerriAI/litellm/pull/29695)
- fix(helm): Enable Backend Deployment to mount Gateway config.yaml by [@&#8203;tin-berri](https://github.com/tin-berri) in [#&#8203;29605](https://github.com/BerriAI/litellm/pull/29605)
- \[internal copy of [#&#8203;29277](https://github.com/BerriAI/litellm/issues/29277)] fix(proxy): add default=None to LiteLLM\_TeamMembership.litellm\_budget\_table by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29684](https://github.com/BerriAI/litellm/pull/29684)
- test: make custom\_tokenizer proxy tests hermetic by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29643](https://github.com/BerriAI/litellm/pull/29643)
- test(proxy): stop running real-DB tests in GitHub Actions unit jobs by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29700](https://github.com/BerriAI/litellm/pull/29700)
- chore(ui): remove the bare-fetch lint rule by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29712](https://github.com/BerriAI/litellm/pull/29712)
- Litellm jwt mapping virtualkeys by [@&#8203;shivamrawat1](https://github.com/shivamrawat1) in [#&#8203;28510](https://github.com/BerriAI/litellm/pull/28510)
- refactor(ui): shared HTTP client + location-pinned fetch() lint rule by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29723](https://github.com/BerriAI/litellm/pull/29723)
- fix(proxy): stop team BYOK model name corruption on model edit by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29731](https://github.com/BerriAI/litellm/pull/29731)
- \[internal copy of [#&#8203;29511](https://github.com/BerriAI/litellm/issues/29511)] feat(guardrails): add sensitive data routing to on-premise models by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29531](https://github.com/BerriAI/litellm/pull/29531)
- fix(proxy/hooks): populate llm\_provider on internal rate-limit errors by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;27707](https://github.com/BerriAI/litellm/pull/27707)
- fix(vertex/anthropic): handle namespace tools and strip client\_metadata for codex compatibility by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29489](https://github.com/BerriAI/litellm/pull/29489)
- Support OAuth M2M for Databricks Apps A2A agents by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29586](https://github.com/BerriAI/litellm/pull/29586)
- fix: small CLAUDE.md nit by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29749](https://github.com/BerriAI/litellm/pull/29749)
- fix(anthropic): route Claude Opus 4.8 through adaptive thinking by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29702](https://github.com/BerriAI/litellm/pull/29702)
- fix(proxy): persist oauth2\_flow on MCP server registration by [@&#8203;michelligabriele](https://github.com/michelligabriele) in [#&#8203;29690](https://github.com/BerriAI/litellm/pull/29690)
- \[internal copy of [#&#8203;27491](https://github.com/BerriAI/litellm/issues/27491)] fix(realtime): Fix Realtime Audio Token Cost Tracking by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29722](https://github.com/BerriAI/litellm/pull/29722)
- fix(galileo): use ingest traces API and standard logging payload by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29651](https://github.com/BerriAI/litellm/pull/29651)
- fix(auth): expand all-team-models sentinel in can\_key\_call\_model for batch validation by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29746](https://github.com/BerriAI/litellm/pull/29746)
- test(vcr): stop refreshing cassette TTL on read so cassettes lapse after 24h by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29784](https://github.com/BerriAI/litellm/pull/29784)
- test(ci): record/replay OpenAI image gen so the spend E2E isn't outage-bound by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29787](https://github.com/BerriAI/litellm/pull/29787)
- fix(ui): route MCP playground auth by oauth2 mode instead of token\_url by [@&#8203;tin-berri](https://github.com/tin-berri) in [#&#8203;29714](https://github.com/BerriAI/litellm/pull/29714)
- refactor(ui): centralize proxy base URL resolution into tested resolver by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29793](https://github.com/BerriAI/litellm/pull/29793)
- Litellm oss staging 050626 by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29774](https://github.com/BerriAI/litellm/pull/29774)
- test(google): add google-genai SDK proxy integration tests by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29781](https://github.com/BerriAI/litellm/pull/29781)
- fix(jwt): use resolved DB user\_id for spend on legacy email match by [@&#8203;milan-berri](https://github.com/milan-berri) in [#&#8203;29217](https://github.com/BerriAI/litellm/pull/29217)
- feat(ui): generate dashboard API types from the proxy OpenAPI spec by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29816](https://github.com/BerriAI/litellm/pull/29816)
- fix(proxy): drop deleted team BYOK model name from team.models by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29820](https://github.com/BerriAI/litellm/pull/29820)
- feat(mcp): per-server env vars with global + per-user scopes by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;28917](https://github.com/BerriAI/litellm/pull/28917)
- refactor(ui): route behavior-preserving networking calls through apiClient by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29806](https://github.com/BerriAI/litellm/pull/29806)
- fix(mcp): persist Tools-tab MCP OAuth token to DB by [@&#8203;tin-berri](https://github.com/tin-berri) in [#&#8203;29809](https://github.com/BerriAI/litellm/pull/29809)
- fix(ui): require new expiration when regenerating an expired key by [@&#8203;milan-berri](https://github.com/milan-berri) in [#&#8203;29838](https://github.com/BerriAI/litellm/pull/29838)
- refactor(ui): route query-building networking calls through apiClient by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29815](https://github.com/BerriAI/litellm/pull/29815)
- Make the image-gen record/replay proxy report cache mode and per-request HIT/MISS by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29802](https://github.com/BerriAI/litellm/pull/29802)
- feat(proxy): hot-reload .env in dev when running with --reload by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29783](https://github.com/BerriAI/litellm/pull/29783)
- fix(ui): stop MCP playground tool calls from sending twice by [@&#8203;tin-berri](https://github.com/tin-berri) in [#&#8203;29821](https://github.com/BerriAI/litellm/pull/29821)
- feat(fal\_ai): add Nano Banana / Gemini 2.5 Flash Image generation support by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29798](https://github.com/BerriAI/litellm/pull/29798)
- Title: Fix managed batch cancel credential resolution by [@&#8203;shivamrawat1](https://github.com/shivamrawat1) in [#&#8203;29734](https://github.com/BerriAI/litellm/pull/29734)
- Title: fix(proxy): resolve vector store file list credentials from team deployments by [@&#8203;shivamrawat1](https://github.com/shivamrawat1) in [#&#8203;29739](https://github.com/BerriAI/litellm/pull/29739)
- refactor: convert AWS and GCP Terraform stacks into reusable modules … by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;28103](https://github.com/BerriAI/litellm/pull/28103)
- chore(ui): build ui for release by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29853](https://github.com/BerriAI/litellm/pull/29853)
- fix(terraform/gcp): prompt for image\_registry in DeployStack one-click by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29852](https://github.com/BerriAI/litellm/pull/29852)
- fix(terraform/gcp): abandon SQL user on destroy by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29855](https://github.com/BerriAI/litellm/pull/29855)
- Extend the record/replay proxy to chat, embeddings, moderations, rerank, and Anthropic by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29847](https://github.com/BerriAI/litellm/pull/29847)
- chore(deps): bump deps by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29860](https://github.com/BerriAI/litellm/pull/29860)
- chore(ci): promote internal staging to main by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29861](https://github.com/BerriAI/litellm/pull/29861)
- fix: 400 on Anthropic context overflow; seed identity on failed auth by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29848](https://github.com/BerriAI/litellm/pull/29848)
- chore(ci): promote internal staging to main by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29862](https://github.com/BerriAI/litellm/pull/29862)
- chore(release): patch v1.89.0-rc.1 with [#&#8203;30064](https://github.com/BerriAI/litellm/issues/30064) (Claude Fable 5) for v1.89.0-rc.2 by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;30143](https://github.com/BerriAI/litellm/pull/30143)

**Full Changelog**: <https://github.com/BerriAI/litellm/compare/v1.88.0...v1.89.0>

### [`v1.88.2`](https://github.com/BerriAI/litellm/releases/tag/v1.88.2)

[Compare Source](https://github.com/BerriAI/litellm/compare/v1.88.1...v1.88.2)

#### Verify Docker Image Signature

All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0).

**Verify using the pinned commit hash (recommended):**

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

```bash
cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
  ghcr.io/berriai/litellm:v1.88.2
```

**Verify using the release tag (convenience):**

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

```bash
cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/v1.88.2/cosign.pub \
  ghcr.io/berriai/litellm:v1.88.2
```

Expected output:

```
The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key
```

***

#### What's Changed

- chore(release): backport Fable 5, batch-file auth, CrowdStrike AIDR, Mantle Responses SigV4, and NetApp streaming-cost fix to stable/1.88.x and cut 1.88.2 by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;30144](https://github.com/BerriAI/litellm/pull/30144)
- chore(release): backport DB-resilience, passthrough, model-info, budget, and deps fixes to stable/1.88.x by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;30408](https://github.com/BerriAI/litellm/pull/30408)

**Full Changelog**: <https://github.com/BerriAI/litellm/compare/v1.88.1...v1.88.2>

### [`v1.88.1`](https://github.com/BerriAI/litellm/releases/tag/v1.88.1)

[Compare Source](https://github.com/BerriAI/litellm/compare/v1.88.0...v1.88.1)

#### Verify Docker Image Signature

All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0).

**Verify using the pinned commit hash (recommended):**

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

```bash
cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
  ghcr.io/berriai/litellm:v1.88.1
```

**Verify using the release tag (convenience):**

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

```bash
cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/v1.88.1/cosign.pub \
  ghcr.io/berriai/litellm:v1.88.1
```

Expected output:

```
The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key
```

***

#### What's Changed

- build(deps): bump pyjwt to 2.13.0 and ws override to 8.20.1 (1.88.x) by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29987](https://github.com/BerriAI/litellm/pull/29987)
- chore(release): bump version to 1.88.1 by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29989](https://github.com/BerriAI/litellm/pull/29989)

**Full Changelog**: <https://github.com/BerriAI/litellm/compare/v1.88.0...v1.88.1>

### [`v1.88.0`](https://github.com/BerriAI/litellm/releases/tag/v1.88.0)

[Compare Source](https://github.com/BerriAI/litellm/compare/v1.87.3...v1.88.0)

#### Verify Docker Image Signature

All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0).

**Verify using the pinned commit hash (recommended):**

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

```bash
cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
  ghcr.io/berriai/litellm:v1.88.0
```

**Verify using the release tag (convenience):**

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

```bash
cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/v1.88.0/cosign.pub \
  ghcr.io/berriai/litellm:v1.88.0
```

Expected output:

```
The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key
```

***

#### What's Changed

- fix(proxy): gate team allowed\_passthrough\_routes to proxy admins by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;28097](https://github.com/BerriAI/litellm/pull/28097)
- fix(tests): stabilize image-edit VCR cassettes to stop live gpt-image-1 spend by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;28110](https://github.com/BerriAI/litellm/pull/28110)
- fix(bedrock/cohere): send embedding\_types as JSON array, not string by [@&#8203;ishaan-berri](https://github.com/ishaan-berri) in [#&#8203;28172](https://github.com/BerriAI/litellm/pull/28172)
- fix(tests): migrate realtime + rerank tests off shut-down upstream models by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28191](https://github.com/BerriAI/litellm/pull/28191)
- fix(caching): replay openai/responses bridge cache hits as chat streams by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28158](https://github.com/BerriAI/litellm/pull/28158)
- Litellm oss staging by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28161](https://github.com/BerriAI/litellm/pull/28161)
- feat(prometheus): add user\_email and user\_alias to user budget metrics by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28155](https://github.com/BerriAI/litellm/pull/28155)
- test(callbacks): harden flaky proxy callback-leak detector by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28195](https://github.com/BerriAI/litellm/pull/28195)
- fix(bedrock): sanitize batch metadata to prevent Pydantic ValidationError by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;28202](https://github.com/BerriAI/litellm/pull/28202)
- fix(deepseek): use native /anthropic/v1/messages endpoint and sanitize tools by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;28200](https://github.com/BerriAI/litellm/pull/28200)
- feat(ui): add Interactions API endpoint to playground with SSE streaming by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28156](https://github.com/BerriAI/litellm/pull/28156)
- fix(proxy): decode bytes and pass-through SSE for Google-native streamGenerateContent ([#&#8203;27444](https://github.com/BerriAI/litellm/issues/27444)) by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28213](https://github.com/BerriAI/litellm/pull/28213)
- refactor(bedrock/sagemaker): switch to lazy loading for response stre… by [@&#8203;harish-berri](https://github.com/harish-berri) in [#&#8203;28189](https://github.com/BerriAI/litellm/pull/28189)
- \[Refactor] UI - Spend Logs: consolidate filter state and extract components by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;25847](https://github.com/BerriAI/litellm/pull/25847)
- fix(tests): replace shut-down gpt-4o-audio-preview with gpt-audio-1.5 by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28281](https://github.com/BerriAI/litellm/pull/28281)
- chore(ci): bump versions by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28287](https://github.com/BerriAI/litellm/pull/28287)
- feat: propagate team\_id and team\_alias to all child OTEL spans by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;28273](https://github.com/BerriAI/litellm/pull/28273)
- Day 0 support : Gemini 3.5 Flash by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28268](https://github.com/BerriAI/litellm/pull/28268)
- Gemini managed agents support by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28270](https://github.com/BerriAI/litellm/pull/28270)
- chore(ci): promote internal staging to main by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28292](https://github.com/BerriAI/litellm/pull/28292)
- feat(gemini): add gemini-3.1-flash-lite model cost map by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28320](https://github.com/BerriAI/litellm/pull/28320)
- fix(spend\_counter): seed Redis counter via SET NX to prevent cross-pod double-seed by [@&#8203;milan-berri](https://github.com/milan-berri) in [#&#8203;27854](https://github.com/BerriAI/litellm/pull/27854)
- fix(proxy): normalize batch file IDs before ManagedObjectTable write by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28339](https://github.com/BerriAI/litellm/pull/28339)
- fix(router): use forwarded model\_id for native Azure container IDs by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;27921](https://github.com/BerriAI/litellm/pull/27921)
- fix(ui): restore log filter loading indicator by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;28282](https://github.com/BerriAI/litellm/pull/28282)
- test(e2e): migrate runner to uv, add All Proxy Models key test by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;28313](https://github.com/BerriAI/litellm/pull/28313)
- feat(ui): team passthrough routes create parity + edit load fix by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;28098](https://github.com/BerriAI/litellm/pull/28098)
- fix(mcp): JWT on tools/list and REST tools/call server resolution by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28227](https://github.com/BerriAI/litellm/pull/28227)
- feat(interactions): migrate to Google Interactions API steps schema (May 2026) by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28153](https://github.com/BerriAI/litellm/pull/28153)
- test(ui-e2e): admin key creation with a specific proxy model by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;28365](https://github.com/BerriAI/litellm/pull/28365)
- fix(vertex\_ai): omit function\_call id on Vertex Gemini 3.5+ tool turns by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28324](https://github.com/BerriAI/litellm/pull/28324)
- feat(mcp): allow native MCP OAuth support for cursor by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28327](https://github.com/BerriAI/litellm/pull/28327)
- fix(interactions): never drop streamed text deltas; always emit terminal completion by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;28394](https://github.com/BerriAI/litellm/pull/28394)
- fix(proxy): expose Prisma idle/connect timeout + extra DB URL params by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;28395](https://github.com/BerriAI/litellm/pull/28395)
- Litellm oss staging 1 by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28337](https://github.com/BerriAI/litellm/pull/28337)
- fix: serialize guardrail\_response to JSON in OTEL traces by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;28362](https://github.com/BerriAI/litellm/pull/28362)
- chore(ci): merge dev branch by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28314](https://github.com/BerriAI/litellm/pull/28314)
- test(realtime): expect session.created as xAI realtime initial event by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28424](https://github.com/BerriAI/litellm/pull/28424)
- feat(tests): behavior-pinning harness + Key Tier-1 matrix by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28321](https://github.com/BerriAI/litellm/pull/28321)
- fix(proxy): hydrate wildcard discovery credentials ([#&#8203;28284](https://github.com/BerriAI/litellm/issues/28284)) - CCI Run by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28419](https://github.com/BerriAI/litellm/pull/28419)
- Litellm oss staging 04 21 2026 2 by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;26569](https://github.com/BerriAI/litellm/pull/26569)
- chore(ci): merge dev branch by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28290](https://github.com/BerriAI/litellm/pull/28290)
- fix(vertex\_gemma): strip `context_management` from request body by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;28438](https://github.com/BerriAI/litellm/pull/28438)
- fix(logging): recalculate cost after router retry failures by [@&#8203;milan-berri](https://github.com/milan-berri) in [#&#8203;28476](https://github.com/BerriAI/litellm/pull/28476)
- fix(otel): emit guardrail span on violation, surface status + categories by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;28364](https://github.com/BerriAI/litellm/pull/28364)
- test(proxy): behavior-pinning matrix for team management endpoints by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28441](https://github.com/BerriAI/litellm/pull/28441)
- test(vertex\_ai): tolerate transient 500 in google maps grounding test by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28503](https://github.com/BerriAI/litellm/pull/28503)
- fix(docker): restore npm to non\_root builder image by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28519](https://github.com/BerriAI/litellm/pull/28519)
- chore(ci): bump deps by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28524](https://github.com/BerriAI/litellm/pull/28524)
- build(deps-dev): bump black to 26.3.1 and apply formatting by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28525](https://github.com/BerriAI/litellm/pull/28525)
- chore(deps): bump deps by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28528](https://github.com/BerriAI/litellm/pull/28528)
- test(e2e): forward LITELLM\_LICENSE to UI e2e proxy by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;28398](https://github.com/BerriAI/litellm/pull/28398)
- Add granian as a ASGI compliant web server. Provider better throughput stability, by [@&#8203;harish-berri](https://github.com/harish-berri) in [#&#8203;26027](https://github.com/BerriAI/litellm/pull/26027)
- Fix conflicts and UI by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28477](https://github.com/BerriAI/litellm/pull/28477)
- Add error\_description and hint for oauth flows by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28471](https://github.com/BerriAI/litellm/pull/28471)
- feat(mcp): Add tool call and tool list support via UI for Oauth mcps by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28454](https://github.com/BerriAI/litellm/pull/28454)
- feat(proxy): persist allowlisted OIDC claims in CLI SSO poll by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28463](https://github.com/BerriAI/litellm/pull/28463)
- fix(responses): use OpenAI SSEDecoder for Responses API streaming by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28566](https://github.com/BerriAI/litellm/pull/28566)
- Litellm oss staging 2 by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28582](https://github.com/BerriAI/litellm/pull/28582)
- \[internal copy of [#&#8203;28269](https://github.com/BerriAI/litellm/issues/28269)] Codex cli jwt team alias by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;28621](https://github.com/BerriAI/litellm/pull/28621)
- fix(check\_licenses): read PEP 639 license-expression metadata by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28529](https://github.com/BerriAI/litellm/pull/28529)
- test(proxy): behavior-pinning matrix for tier-2/3 key + team management endpoints by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28620](https://github.com/BerriAI/litellm/pull/28620)
- chore(test): remove dead old Playwright e2e suite by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;28632](https://github.com/BerriAI/litellm/pull/28632)
- fix(sagemaker): send native Cohere embed payload to Cohere SageMaker endpoints by [@&#8203;milan-berri](https://github.com/milan-berri) in [#&#8203;28613](https://github.com/BerriAI/litellm/pull/28613)
- style: apply black formatting to fix lint CI (LIT-3274) ([#&#8203;28639](https://github.com/BerriAI/litellm/issues/28639)) by [@&#8203;krrish-berri-2](https://github.com/krrish-berri-2) in [#&#8203;28641](https://github.com/BerriAI/litellm/pull/28641)
- fix(bedrock): decouple STS region from Bedrock aws\_region\_name by [@&#8203;milan-berri](https://github.com/milan-berri) in [#&#8203;28245](https://github.com/BerriAI/litellm/pull/28245)
- test(streaming): tolerate Vertex 429 wrapped in MidStreamFallbackError by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28669](https://github.com/BerriAI/litellm/pull/28669)
- feat(guardrails): add Microsoft Purview DLP guardrail by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;24966](https://github.com/BerriAI/litellm/pull/24966)
- fix(mcp): forward upstream initialize instructions on cold gateway init by [@&#8203;milan-berri](https://github.com/milan-berri) in [#&#8203;28231](https://github.com/BerriAI/litellm/pull/28231)
- chore(ci): promote internal staging to main by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28680](https://github.com/BerriAI/litellm/pull/28680)
- CI: copy of [#&#8203;25177](https://github.com/BerriAI/litellm/issues/25177) (OCI GenAI: embeddings, streaming/reasoning fixes, model catalog) by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;28223](https://github.com/BerriAI/litellm/pull/28223)
- Encrypt callback\_vars in key/team metadata in DB by [@&#8203;Michael-RZ-Berri](https://github.com/Michael-RZ-Berri) in [#&#8203;27141](https://github.com/BerriAI/litellm/pull/27141)
- perf: reduce per-request and per-chunk overhead across Anthropic streaming hot paths by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;28289](https://github.com/BerriAI/litellm/pull/28289)
- feat(azure): add Speech STT config support by [@&#8203;ishaan-berri](https://github.com/ishaan-berri) in [#&#8203;27482](https://github.com/BerriAI/litellm/pull/27482)
- test(proxy): phase-4 payload behavior pinning for tier-2/3 key + team management endpoints by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28681](https://github.com/BerriAI/litellm/pull/28681)
- feat(prometheus): emit per-token-type detail metrics (LIT-3220) ([#&#8203;28372](https://github.com/BerriAI/litellm/issues/28372)) by [@&#8203;ishaan-berri](https://github.com/ishaan-berri) in [#&#8203;28378](https://github.com/BerriAI/litellm/pull/28378)
- fix(otel): stamp http.response.status\_code on all error responses by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;28405](https://github.com/BerriAI/litellm/pull/28405)
- chore(ui): build ui by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28707](https://github.com/BerriAI/litellm/pull/28707)
- fix(helm): drop main- prefix from default image tag by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28710](https://github.com/BerriAI/litellm/pull/28710)
- test(model\_prices): allow audio\_transcription\_config in schema by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28708](https://github.com/BerriAI/litellm/pull/28708)
- chore(ci): promote internal staging to main by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28709](https://github.com/BerriAI/litellm/pull/28709)
- fix(team): refresh team cache on team\_model\_add/delete (LIT-3244) by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28683](https://github.com/BerriAI/litellm/pull/28683)
- fix(ui/add-model): stop vertex\_ai-anthropic\_models from leaking into Anthropic dropdown by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;28723](https://github.com/BerriAI/litellm/pull/28723)
- Fix spend logs v2 route permissions by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;28705](https://github.com/BerriAI/litellm/pull/28705)
- fix(proxy): Bedrock Knowledge Base pass-through: preserve SigV4 headers and signed request body by [@&#8203;milan-berri](https://github.com/milan-berri) in [#&#8203;27526](https://github.com/BerriAI/litellm/pull/27526)
- chore(tests): migrate Bedrock CI to AWS account [`9412775`](https://github.com/BerriAI/litellm/commit/941277531214) by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;28728](https://github.com/BerriAI/litellm/pull/28728)
- fix(otel): export SERVER span on management-endpoint success without http\_request by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;28794](https://github.com/BerriAI/litellm/pull/28794)
- chore(ci): merge dev branch by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28801](https://github.com/BerriAI/litellm/pull/28801)
- chore(ci): merge dev branch by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28657](https://github.com/BerriAI/litellm/pull/28657)
- fix(ui): show 2-decimal precision for max\_budget on key overview by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;28809](https://github.com/BerriAI/litellm/pull/28809)
- feat(proxy): allow `llm_api_routes` virtual keys to list MCP servers by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;28442](https://github.com/BerriAI/litellm/pull/28442)
- chore(ci): merge dev branch by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28807](https://github.com/BerriAI/litellm/pull/28807)
- fix(team): keep team\_alias cache in sync on \_cache\_team\_object writes by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28737](https://github.com/BerriAI/litellm/pull/28737)
- chore(ci): merge dev branch by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28822](https://github.com/BerriAI/litellm/pull/28822)
- ci: daily oss-agent-shin canonical branch by [@&#8203;ishaan-berri](https://github.com/ishaan-berri) in [#&#8203;28829](https://github.com/BerriAI/litellm/pull/28829)
- test(proxy): add harness for proxy\_server.py behavior-pinning by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28827](https://github.com/BerriAI/litellm/pull/28827)
- feat(openai): apply regional-processing cost uplift for EU/US data residency by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;28626](https://github.com/BerriAI/litellm/pull/28626)
- chore(admin-ui): regenerate static export with trailingSlash: true by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;28112](https://github.com/BerriAI/litellm/pull/28112)
- fix(azure): preserve AD token refresh in v1 OpenAI client path by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;28627](https://github.com/BerriAI/litellm/pull/28627)
- fix(ui): route API Reference back to query-param page by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;28726](https://github.com/BerriAI/litellm/pull/28726)
- fix(model-edit): allow clearing custom pricing on wildcard models by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;28719](https://github.com/BerriAI/litellm/pull/28719)
- fix(tests/vcr): make Redis cassette cache replay deterministically (zero VCR misses on consecutive runs) by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;28826](https://github.com/BerriAI/litellm/pull/28826)
- fix(proxy): strip LiteLLM policy tracking from OpenAI batch metadata by [@&#8203;shivamrawat1](https://github.com/shivamrawat1) in [#&#8203;28425](https://github.com/BerriAI/litellm/pull/28425)
- Litellm OpenAI double prefix bug by [@&#8203;shivamrawat1](https://github.com/shivamrawat1) in [#&#8203;28661](https://github.com/BerriAI/litellm/pull/28661)
- Litellm oss staging 250526 by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28770](https://github.com/BerriAI/litellm/pull/28770)
- fix(bedrock): align toolUse/toolSpec names and allow hyphens by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28874](https://github.com/BerriAI/litellm/pull/28874)
- fix(realtime): send TEXT frames and valid guardrail session.update by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28848](https://github.com/BerriAI/litellm/pull/28848)
- fix(mcp): extend key access-group union to MCP servers by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;28890](https://github.com/BerriAI/litellm/pull/28890)
- fix(galileo): support hosted v2 spans API and string output extraction by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28771](https://github.com/BerriAI/litellm/pull/28771)
- fix(proxy): exclude proxy\_server\_request from its own body snapshot by [@&#8203;michelligabriele](https://github.com/michelligabriele) in [#&#8203;28618](https://github.com/BerriAI/litellm/pull/28618)
- \[Feat] Add tool calling support for gemini and vertex ai live api by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;26590](https://github.com/BerriAI/litellm/pull/26590)
- refactor(ui): remove dead App Router scaffolding in (dashboard)/\* by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;28891](https://github.com/BerriAI/litellm/pull/28891)
- fix(docker): use system Node in componentized builders + retry apk add by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;28888](https://github.com/BerriAI/litellm/pull/28888)
- docs(agents): require consent before writing new third-party names by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;28908](https://github.com/BerriAI/litellm/pull/28908)
- refactor(ui): extract auth state into AuthContext by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;28910](https://github.com/BerriAI/litellm/pull/28910)
- fix(mcp): resolve team.access\_group\_ids → MCP servers by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;28997](https://github.com/BerriAI/litellm/pull/28997)
- test(ui): e2e cover team model edit + admin identity in navbar by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;28652](https://github.com/BerriAI/litellm/pull/28652)
- test(e2e): cover add-fallback flow in Router Settings by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29069](https://github.com/BerriAI/litellm/pull/29069)
- test(e2e): cover Team-BYOK add-model flow as proxy admin by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29068](https://github.com/BerriAI/litellm/pull/29068)
- fix(containers): record ownership for service-account keys + fix Prisma Json serialization by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28990](https://github.com/BerriAI/litellm/pull/28990)
- test(e2e): cover add-MCP-server flow via discovery → custom form by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29070](https://github.com/BerriAI/litellm/pull/29070)
- test(e2e): cover AI Hub make-public flow and public model\_hub\_table by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29071](https://github.com/BerriAI/litellm/pull/29071)
- \[internal copy of [#&#8203;28877](https://github.com/BerriAI/litellm/issues/28877)] feat: add support for claude code goal mode for bedrock opus output config by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;28898](https://github.com/BerriAI/litellm/pull/28898)
- feat(guardrails): wire apply\_guardrail into proxy logging callbacks by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28970](https://github.com/BerriAI/litellm/pull/28970)
- chore(ci): merge dev brach by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29192](https://github.com/BerriAI/litellm/pull/29192)
- perf(streaming): cut per-chunk overhead \~30% on Anthropic + Bedrock hot path by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;28720](https://github.com/BerriAI/litellm/pull/28720)
- fix(proxy): enforce tag budgets for key-level tags by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29108](https://github.com/BerriAI/litellm/pull/29108)
- fix(vertex-ai): use DB credentials in video handlers + implement Veo video edit by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29098](https://github.com/BerriAI/litellm/pull/29098)
- fix(datadog): drain cost-management queue + opt-in FinOps tag allowlist by [@&#8203;michelligabriele](https://github.com/michelligabriele) in [#&#8203;28487](https://github.com/BerriAI/litellm/pull/28487)
- feat(helm): split per-component ServiceAccounts for gateway, backend, and UI by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;28712](https://github.com/BerriAI/litellm/pull/28712)
- chore(ci): bump deps ([#&#8203;29208](https://github.com/BerriAI/litellm/issues/29208)) by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29226](https://github.com/BerriAI/litellm/pull/29226)
- fix(tests/vcr): mint Google OAuth tokens live to prevent stale-token replay by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29229](https://github.com/BerriAI/litellm/pull/29229)
- chore(cookbook): bump Go directive to 1.26.3 in gollem example by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29234](https://github.com/BerriAI/litellm/pull/29234)
- chore(ci): bump version by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29242](https://github.com/BerriAI/litellm/pull/29242)
- feat(anthropic): add Claude Opus 4.8 and prune reasoning-effort flags by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29238](https://github.com/BerriAI/litellm/pull/29238)
- chore(ci): promote internal staging to main by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29243](https://github.com/BerriAI/litellm/pull/29243)
- fix(ci): restore real Bedrock batch S3 bucket/role in oai\_misc\_config by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29245](https://github.com/BerriAI/litellm/pull/29245)
- fix(guardrails): persist disable\_global\_guardrails on keys by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29233](https://github.com/BerriAI/litellm/pull/29233)
- test(e2e): cover Team Admin view + member + key flows by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29072](https://github.com/BerriAI/litellm/pull/29072)
- docs: hand-written CLAUDE.md; remove AGENTS.md, point GEMINI.md at it by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29252](https://github.com/BerriAI/litellm/pull/29252)
- fix(teams): expose keys\_count on /v2/team/list and wire UI Resources badge by [@&#8203;michelligabriele](https://github.com/michelligabriele) in [#&#8203;28502](https://github.com/BerriAI/litellm/pull/28502)
- fix(anthropic): stop injecting unsupported output\_config.effort=xhigh for Claude Code on Sonnet/Opus 4.6 by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29304](https://github.com/BerriAI/litellm/pull/29304)
- test(e2e): cover Internal Viewer nav, key, and team-info gating by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29075](https://github.com/BerriAI/litellm/pull/29075)
- test(e2e): cover Internal User key modal, team info, key page by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29074](https://github.com/BerriAI/litellm/pull/29074)
- test(e2e): cover navbar Logout flow as proxy admin by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29076](https://github.com/BerriAI/litellm/pull/29076)
- fix(mcp): resolve key.access\_group\_ids → MCP servers (ungated) by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29195](https://github.com/BerriAI/litellm/pull/29195)
- fix(router): enforce deployment budgets for dynamically added models by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29273](https://github.com/BerriAI/litellm/pull/29273)
- fix(proxy): map stripped batch body.model to proxy alias for auth by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29264](https://github.com/BerriAI/litellm/pull/29264)
- feat(mcp): support stateless and stateful clients via session-id routing by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;26857](https://github.com/BerriAI/litellm/pull/26857)
- fix(bedrock): support tool search results + chat annotations by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29120](https://github.com/BerriAI/litellm/pull/29120)
- fix(mcp): ignore stale ids on key save by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29128](https://github.com/BerriAI/litellm/pull/29128)
- feat(a2a): well-known agent-card discovery + LangGraph Platform mode by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28860](https://github.com/BerriAI/litellm/pull/28860)
- fix(proxy): link passthrough success spans to the SERVER root OTEL span by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29315](https://github.com/BerriAI/litellm/pull/29315)
- \[internal copy of [#&#8203;29089](https://github.com/BerriAI/litellm/issues/29089)] fix: duplicate claude code traces by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29311](https://github.com/BerriAI/litellm/pull/29311)
- feat(otel): typed semconv-aligned OpenTelemetry instrumentation by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;28909](https://github.com/BerriAI/litellm/pull/28909)
- tests(proxy\_server): surface current behavior in tests by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29309](https://github.com/BerriAI/litellm/pull/29309)
- test(e2e): cover Internal User create-key flow when in no teams by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29083](https://github.com/BerriAI/litellm/pull/29083)
- test(e2e): assert internal-user navbar identity is scoped to that user by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29077](https://github.com/BerriAI/litellm/pull/29077)
- feat(otel): add team\_metadata, http.route, and model names to inference spans by [@&#8203;yassin-berriai](https://github.com/yassin-berriai) in [#&#8203;29319](https://github.com/BerriAI/litellm/pull/29319)
- feat(context\_management): compact\_20260112 polyfill for non-Anthropic providers by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;28868](https://github.com/BerriAI/litellm/pull/28868)
- feat(enterprise): add RESEND\_FROM\_EMAIL for self-hosted Resend sends by [@&#8203;shivamrawat1](https://github.com/shivamrawat1) in [#&#8203;28830](https://github.com/BerriAI/litellm/pull/28830)
- Revert Bedrock CI back to the reactivated AWS account ([`8886022`](https://github.com/BerriAI/litellm/commit/888602223428)) by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29326](https://github.com/BerriAI/litellm/pull/29326)
- fix(mcp): preserve source\_url in GET /v1/mcp/server list responses by [@&#8203;shivamrawat1](https://github.com/shivamrawat1) in [#&#8203;29249](https://github.com/BerriAI/litellm/pull/29249)
- fix(mcp): preserve omitted fields on PUT /v1/mcp/server partial updates by [@&#8203;shivamrawat1](https://github.com/shivamrawat1) in [#&#8203;29253](https://github.com/BerriAI/litellm/pull/29253)
- fix(ci): make litellm\_internal\_staging green (logging test + Bedrock Opus 4.7 self-heal) by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29344](https://github.com/BerriAI/litellm/pull/29344)
- refactor(proxy/auth): normalize Bearer prefix in safe-hash helper by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29343](https://github.com/BerriAI/litellm/pull/29343)
- test(reasoning-effort-grid): cover Claude Opus 4.8 across provider routes by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29327](https://github.com/BerriAI/litellm/pull/29327)
- fix(guardrails): return HTTP 400 for litellm content filter blocks by [@&#8203;shivamrawat1](https://github.com/shivamrawat1) in [#&#8203;28418](https://github.com/BerriAI/litellm/pull/28418)
- fix(proxy): restrict vector store index create/delete to proxy admins by [@&#8203;shivamrawat1](https://github.com/shivamrawat1) in [#&#8203;29202](https://github.com/BerriAI/litellm/pull/29202)
- feat(pass\_through): extend passthrough\_managed\_object\_ids to Azure by [@&#8203;Sameerlite](https://github.com/Sameerlite) in [#&#8203;29160](https://github.com/BerriAI/litellm/pull/29160)
- fix(proxy): enforce allowed\_passthrough\_routes for auth=true pass-thr… by [@&#8203;shivamrawat1](https://github.com/shivamrawat1) in [#&#8203;29256](https://github.com/BerriAI/litellm/pull/29256)
- feat(mcp/auth): additive key access-group grants + opt-in member assignment by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29313](https://github.com/BerriAI/litellm/pull/29313)
- fix(reset\_budget): write only {spend, budget\_reset\_at} and stop pre-zeroing counter by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29358](https://github.com/BerriAI/litellm/pull/29358)
- test(e2e): cover PROXY\_LOGOUT\_URL redirect on Logout by [@&#8203;ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#&#8203;29080](https://github.com/BerriAI/litellm/pull/29080)
- fix(ui): break logout redirect loop across dev and proxy origins by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29360](https://github.com/BerriAI/litellm/pull/29360)
- fix(openai-moderation): wire streaming flags through to unified dispatcher by [@&#8203;michelligabriele](https://github.com/michelligabriele) in [#&#8203;27324](https://github.com/BerriAI/litellm/pull/27324)
- chore(ci): build ui by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29366](https://github.com/BerriAI/litellm/pull/29366)
- fix(v3 limiter): cap no-max\_tokens TPM floor at smallest configured limit by [@&#8203;michelligabriele](https://github.com/michelligabriele) in [#&#8203;28805](https://github.com/BerriAI/litellm/pull/28805)
- fix(e2e): tolerate trailing slash in SERVER\_ROOT\_PATH login redirect by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29369](https://github.com/BerriAI/litellm/pull/29369)
- chore(deps): bump deps by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29373](https://github.com/BerriAI/litellm/pull/29373)
- chore(ci): promote internal staging to main by [@&#8203;yuneng-berri](https://github.com/yuneng-berri) in [#&#8203;29372](https://github.com/BerriAI/litellm/pull/29372)
- chore(release): patch v1.88.0-rc.1 with four staged fixes by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29632](https://github.com/BerriAI/litellm/pull/29632)
- chore(release): patch v1.88.0-rc.1 with [#&#8203;29612](https://github.com/BerriAI/litellm/issues/29612) (session-token budget-ceiling exemption) by [@&#8203;mateo-berri](https://github.com/mateo-berri) in [#&#8203;29637](https://github.com/BerriAI/litellm/pull/29637)
- fix(key\_generate): harden GHSA-q775 …
fzowl pushed a commit to fzowl/litellm that referenced this pull request Jun 24, 2026
* feat(a2a): add watsonx Orchestrate agent provider

Bridge A2A message/send to WXO runs API (CP4D and IBM Cloud IAM auth),
with dashboard agent type metadata and unit tests for transformations.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(a2a): use shared httpx client and cache WXO auth tokens

Route WXO streaming through get_async_httpx_client (TLS verification
enabled). Cache bearer tokens with TTL buffer. Extract A2A reply text via
a dedicated helper instead of hard-coded JSON paths.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(a2a): treat CP4D token expiration as absolute Unix time

CP4D /authorize returns expiration as epoch seconds, not TTL. Compute
remaining lifetime against wall clock so cached tokens refresh before expiry.

Co-authored-by: Cursor <cursoragent@cursor.com>

* style(a2a): black-format watsonx orchestrate handler for CI py312

Co-authored-by: Cursor <cursoragent@cursor.com>

* Fix watsonx orchestrate edge cases

* Fix WXO streaming fallback error handling

* Fix watsonx orchestrate run completion handling

* fix(a2a): make WXO username optional in agent create UI

Username is only required for cp4d auth; ibm_cloud uses api_key alone.
Backend still validates username when auth_mode is cp4d.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(a2a): align WXO dashboard field test with optional username

Username is not required in agent_create_fields.json; backend validates
for cp4d auth_mode only.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(a2a): send Accept header on WXO streaming run request

* fix(a2a/wxo): scope streaming transport fallback to initial POST only

Narrow the httpx.TransportError fallback in handle_streaming so it only
covers the initial POST /runs/stream. Errors during polling or SSE
consumption now propagate instead of triggering handle_non_streaming,
which would have submitted a duplicate WXO run for the same request.

* refactor(a2a/wxo): type run-param extraction and use text response_type

Return a typed WXORequestParams NamedTuple from _extract_litellm_params
instead of a positional tuple so call sites read params by name, and send
the user message with response_type 'text' so the run body is valid across
all WXO agent configurations rather than the search-specific type.

* fix(a2a/wxo): evict expired token cache entries and raise asyncio.TimeoutError on poll timeout

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants