feat(a2a): watsonx Orchestrate agent provider - #29410
Conversation
Bridge A2A message/send to WXO runs API (CP4D and IBM Cloud IAM auth), with dashboard agent type metadata and unit tests for transformations. Co-authored-by: Cursor <cursoragent@cursor.com>
Greptile SummaryThis PR adds a
Confidence Score: 5/5The PR is safe to merge; the new provider is self-contained, uses litellm's shared HTTP infrastructure, and does not touch any existing request paths. All three major issues flagged in the previous review round have been corrected: the streaming path now uses get_async_httpx_client instead of a raw httpx.AsyncClient(verify=False), the try/except httpx.TransportError block covers only the initial POST, and the CP4D expiration field is now correctly treated as an absolute Unix timestamp. The remaining findings are style-level redundancies that do not affect correctness. No files require special attention; handler.py carries the most complexity but the logic is well-tested and the auth paths are correct.
|
| Filename | Overview |
|---|---|
| litellm/a2a_protocol/providers/watsonx_orchestrate/handler.py | Core handler with auth, polling, SSE accumulation, and streaming fallback; previously flagged issues have been addressed; explicit response.raise_for_status() after client.post() is redundant since AsyncHTTPHandler.post already calls it internally |
| litellm/a2a_protocol/providers/watsonx_orchestrate/transformation.py | Transformation helpers for A2A↔WXO mapping and fake streaming; well-structured with multiple fallback paths for text extraction |
| litellm/a2a_protocol/providers/config_manager.py | Registers watsonx_orchestrate provider with lazy import; follows the same pattern as BedrockAgentCore |
| litellm/proxy/public_endpoints/agent_create_fields.json | Adds watsonx Orchestrate dashboard agent type; field definitions match the handler's _extract_litellm_params requirements |
| tests/test_litellm/a2a_protocol/providers/watsonx_orchestrate/test_watsonx_orchestrate_transformation.py | Good mock-only test coverage for transformation helpers, token TTL, cache behavior, SSE accumulation, and streaming fallback/poll paths; assertions are correct |
Reviews (11): Last reviewed commit: "fix(a2a/wxo): evict expired token cache ..." | Re-trigger Greptile
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
PR overviewAll previously flagged issues have been addressed. No open security concerns remain on this pull request. Security reviewNo open security issues remain on this pull request. Fixed/addressed: 1 · PR risk: 0/10 |
Route WXO streaming through get_async_httpx_client (TLS verification enabled). Cache bearer tokens with TTL buffer. Extract A2A reply text via a dedicated helper instead of hard-coded JSON paths. Co-authored-by: Cursor <cursoragent@cursor.com>
CP4D /authorize returns expiration as epoch seconds, not TTL. Compute remaining lifetime against wall clock so cached tokens refresh before expiry. Co-authored-by: Cursor <cursoragent@cursor.com>
|
@greptileai Addressed P1: CP4D |
Co-authored-by: Cursor <cursoragent@cursor.com>
|
bugbot run |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 3 potential issues.
Autofix Details
Bugbot Autofix prepared fixes for all 3 issues found in the latest run.
- ✅ Fixed: Text extraction elif matches non-text kind parts
- A2A text extraction now only includes parts whose kind is text, missing, or empty, with regression coverage for non-text parts carrying text fields.
- ✅ Fixed: SSE parser crashes on non-dict JSON events
- WXO result text extraction now returns an empty string for non-dict JSON values, so SSE accumulation ignores keepalive-style events without aborting.
- ✅ Fixed: Token cache minimum can serve expired tokens
- Token cache expiry now floors at zero after the safety buffer, preventing short-lived tokens from being served past their actual expiration.
You can send follow-ups to the cloud agent here.
|
|
|
bugbot run |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
Autofix Details
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Broad exception catch silently swallows streaming errors causing double requests
- Restricted the WXO streaming fallback to transport-level failures so HTTP/status, JSON parsing, and other unexpected errors propagate without a second request.
Preview (3fcbc4ceac)
diff --git a/litellm/a2a_protocol/providers/config_manager.py b/litellm/a2a_protocol/providers/config_manager.py
--- a/litellm/a2a_protocol/providers/config_manager.py
+++ b/litellm/a2a_protocol/providers/config_manager.py
@@ -48,4 +48,11 @@
return BedrockAgentCoreA2AConfig()
+ if custom_llm_provider == "watsonx_orchestrate":
+ from litellm.a2a_protocol.providers.watsonx_orchestrate.config import (
+ WatsonxOrchestrateA2AConfig,
+ )
+
+ return WatsonxOrchestrateA2AConfig()
+
return None
diff --git a/litellm/a2a_protocol/providers/watsonx_orchestrate/__init__.py b/litellm/a2a_protocol/providers/watsonx_orchestrate/__init__.py
new file mode 100644
--- /dev/null
+++ b/litellm/a2a_protocol/providers/watsonx_orchestrate/__init__.py
@@ -1,0 +1,3 @@
+"""
+IBM watsonx Orchestrate (WXO) A2A provider.
+"""
diff --git a/litellm/a2a_protocol/providers/watsonx_orchestrate/config.py b/litellm/a2a_protocol/providers/watsonx_orchestrate/config.py
new file mode 100644
--- /dev/null
+++ b/litellm/a2a_protocol/providers/watsonx_orchestrate/config.py
@@ -1,0 +1,55 @@
+"""
+A2A provider configuration for IBM watsonx Orchestrate (WXO).
+"""
+
+from typing import Any, AsyncIterator, Dict, Optional
+
+from litellm.a2a_protocol.providers.base import BaseA2AProviderConfig
+from litellm.a2a_protocol.providers.watsonx_orchestrate.handler import (
+ WatsonxOrchestrateHandler,
+)
+
+
+class WatsonxOrchestrateA2AConfig(BaseA2AProviderConfig):
+ """A2A bridge for IBM watsonx Orchestrate (REST runs API + poll/SSE)."""
+
+ async def handle_non_streaming(
+ self,
+ request_id: str,
+ params: Dict[str, Any],
+ api_base: Optional[str] = None,
+ **kwargs: Any,
+ ) -> Dict[str, Any]:
+ """Handle a non-streaming A2A request via WXO runs API."""
+ litellm_params = kwargs.get("litellm_params")
+ if not litellm_params:
+ raise ValueError(
+ "litellm_params is required for WatsonxOrchestrateA2AConfig "
+ "(must contain cp4d_host, instance_id, wxo_agent_id, api_key)"
+ )
+ return await WatsonxOrchestrateHandler.handle_non_streaming(
+ request_id=request_id,
+ params=params,
+ litellm_params=litellm_params,
+ )
+
+ async def handle_streaming(
+ self,
+ request_id: str,
+ params: Dict[str, Any],
+ api_base: Optional[str] = None,
+ **kwargs: Any,
+ ) -> AsyncIterator[Dict[str, Any]]:
+ """Handle a streaming A2A request via WXO streaming runs API."""
+ litellm_params = kwargs.get("litellm_params")
+ if not litellm_params:
+ raise ValueError(
+ "litellm_params is required for WatsonxOrchestrateA2AConfig "
+ "(must contain cp4d_host, instance_id, wxo_agent_id, api_key)"
+ )
+ async for chunk in WatsonxOrchestrateHandler.handle_streaming(
+ request_id=request_id,
+ params=params,
+ litellm_params=litellm_params,
+ ):
+ yield chunk
diff --git a/litellm/a2a_protocol/providers/watsonx_orchestrate/handler.py b/litellm/a2a_protocol/providers/watsonx_orchestrate/handler.py
new file mode 100644
--- /dev/null
+++ b/litellm/a2a_protocol/providers/watsonx_orchestrate/handler.py
@@ -1,0 +1,360 @@
+"""
+Handler for IBM watsonx Orchestrate (WXO) agent provider.
+"""
+
+import asyncio
+import hashlib
+import json
+import time
+from typing import Any, AsyncIterator, Dict, Optional, Tuple, cast
+
+import httpx
+
+from litellm._logging import verbose_logger
+from litellm.a2a_protocol.providers.watsonx_orchestrate.transformation import (
+ WatsonxOrchestrateTransformation,
+)
+from litellm.llms.custom_httpx.http_handler import (
+ AsyncHTTPHandler,
+ get_async_httpx_client,
+)
+from litellm.types.llms.custom_http import httpxSpecialProvider
+
+_IBM_CLOUD_IAM_URL = "https://iam.cloud.ibm.com/identity/token"
+_POLL_INTERVAL_S = 2.0
+_MAX_POLL_ATTEMPTS = 90
+_TOKEN_CACHE_TTL_BUFFER_S = 60
+_token_cache: Dict[str, Tuple[str, float]] = {}
+
+
+class WatsonxOrchestrateHandler:
+ @staticmethod
+ def _http_client(timeout: float = 90.0) -> AsyncHTTPHandler:
+ return get_async_httpx_client(
+ llm_provider=cast(Any, httpxSpecialProvider.A2AProvider),
+ params={"timeout": timeout},
+ )
+
+ @staticmethod
+ def _token_cache_key(
+ auth_mode: str,
+ cp4d_host: str,
+ api_key: str,
+ username: Optional[str],
+ ) -> str:
+ material = f"{auth_mode}:{cp4d_host}:{username or ''}:{api_key}"
+ return hashlib.sha256(material.encode()).hexdigest()
+
+ @staticmethod
+ def _cp4d_token_ttl_seconds(
+ expiration: Any, now_wall: Optional[float] = None
+ ) -> int:
+ # CP4D returns expiration as absolute Unix epoch seconds, not a duration.
+ expires_at = int(expiration)
+ wall = now_wall if now_wall is not None else time.time()
+ return max(expires_at - int(wall), 0)
+
+ @staticmethod
+ async def _get_bearer_token(
+ cp4d_host: str,
+ auth_mode: str,
+ api_key: str,
+ username: Optional[str] = None,
+ client: Optional[AsyncHTTPHandler] = None,
+ ) -> str:
+ cache_key = WatsonxOrchestrateHandler._token_cache_key(
+ auth_mode, cp4d_host, api_key, username
+ )
+ now = time.monotonic()
+ cached = _token_cache.get(cache_key)
+ if cached and cached[1] > now:
+ return cached[0]
+
+ if client is None:
+ client = WatsonxOrchestrateHandler._http_client(timeout=30.0)
+
+ if auth_mode == "ibm_cloud":
+ response = await client.post(
+ _IBM_CLOUD_IAM_URL,
+ data={
+ "grant_type": "urn:ibm:params:oauth:grant-type:apikey",
+ "apikey": api_key,
+ },
+ headers={"Content-Type": "application/x-www-form-urlencoded"},
+ )
+ response.raise_for_status()
+ payload = response.json()
+ token = str(payload["access_token"])
+ ttl_s = int(payload.get("expires_in", 3600))
+ else:
+ if not username:
+ raise ValueError(
+ "'username' is required in litellm_params when auth_mode='cp4d'"
+ )
+ token_url = f"{cp4d_host.rstrip('/')}/icp4d-api/v1/authorize"
+ response = await client.post(
+ token_url,
+ json={"username": username, "api_key": api_key},
+ headers={"Content-Type": "application/json"},
+ )
+ response.raise_for_status()
+ payload = response.json()
+ token = str(payload["token"])
+ expiration = payload.get("expiration")
+ if expiration is None:
+ ttl_s = 3600
+ else:
+ ttl_s = WatsonxOrchestrateHandler._cp4d_token_ttl_seconds(expiration)
+
+ expires_at = now + max(ttl_s - _TOKEN_CACHE_TTL_BUFFER_S, 0)
+ _token_cache[cache_key] = (token, expires_at)
+ return token
+
+ @staticmethod
+ async def _poll_run(
+ base_url: str,
+ run_id: str,
+ auth_headers: Dict[str, str],
+ client: AsyncHTTPHandler,
+ max_attempts: int = _MAX_POLL_ATTEMPTS,
+ interval_s: float = _POLL_INTERVAL_S,
+ ) -> Dict[str, Any]:
+ url = f"{base_url}/v1/orchestrate/runs/{run_id}"
+
+ for attempt in range(max_attempts):
+ await asyncio.sleep(interval_s)
+ response = await client.get(url, headers=auth_headers)
+ response.raise_for_status()
+ result: Dict[str, Any] = response.json()
+ status = result.get("status", "")
+ verbose_logger.debug(
+ f"WXO: Poll {attempt + 1}/{max_attempts} run='{run_id}' status='{status}'"
+ )
+ if status in WatsonxOrchestrateTransformation.TERMINAL_STATES:
+ return result
+
+ raise TimeoutError(
+ f"WXO run '{run_id}' did not reach a terminal state after "
+ f"{max_attempts * interval_s:.0f}s"
+ )
+
+ @staticmethod
+ async def _accumulate_wxo_sse_text(response: Any) -> str:
+ accumulated_text = ""
+ async for line in response.aiter_lines():
+ if not line.startswith("data:"):
+ continue
+ data_str = line[5:].strip()
+ if not data_str or data_str == "[DONE]":
+ continue
+ try:
+ event = json.loads(data_str)
+ except json.JSONDecodeError:
+ continue
+ chunk_text = WatsonxOrchestrateTransformation.extract_text_from_wxo_result(
+ event
+ )
+ if chunk_text:
+ accumulated_text += chunk_text
+ return accumulated_text
+
+ @staticmethod
+ def _extract_litellm_params(litellm_params: Dict[str, Any]) -> tuple:
+ cp4d_host = litellm_params.get("cp4d_host") or ""
+ instance_id = litellm_params.get("instance_id") or ""
+ wxo_agent_id = litellm_params.get("wxo_agent_id") or ""
+ api_key = litellm_params.get("api_key") or ""
+ username = litellm_params.get("username") or None
+ auth_mode = litellm_params.get("auth_mode") or "cp4d"
+ thread_id = litellm_params.get("thread_id") or None
+
+ if not cp4d_host:
+ raise ValueError("'cp4d_host' is required in litellm_params for WXO agents")
+ if not instance_id:
+ raise ValueError(
+ "'instance_id' is required in litellm_params for WXO agents"
+ )
+ if not wxo_agent_id:
+ raise ValueError(
+ "'wxo_agent_id' is required in litellm_params for WXO agents"
+ )
+ if not api_key:
+ raise ValueError("'api_key' is required in litellm_params for WXO agents")
+
+ return (
+ cp4d_host,
+ instance_id,
+ wxo_agent_id,
+ api_key,
+ username,
+ auth_mode,
+ thread_id,
+ )
+
+ @staticmethod
+ async def handle_non_streaming(
+ request_id: str,
+ params: Dict[str, Any],
+ litellm_params: Dict[str, Any],
+ ) -> Dict[str, Any]:
+ (
+ cp4d_host,
+ instance_id,
+ wxo_agent_id,
+ api_key,
+ username,
+ auth_mode,
+ thread_id,
+ ) = WatsonxOrchestrateHandler._extract_litellm_params(litellm_params)
+
+ client = WatsonxOrchestrateHandler._http_client(timeout=90.0)
+ token = await WatsonxOrchestrateHandler._get_bearer_token(
+ cp4d_host=cp4d_host,
+ auth_mode=auth_mode,
+ api_key=api_key,
+ username=username,
+ client=client,
+ )
+ base_url = WatsonxOrchestrateTransformation.get_api_base_url(
+ cp4d_host, instance_id
+ )
+ auth_headers = {
+ "Authorization": f"Bearer {token}",
+ "Content-Type": "application/json",
+ "Accept": "application/json",
+ }
+
+ text = WatsonxOrchestrateTransformation.extract_text_from_a2a_params(params)
+ body = WatsonxOrchestrateTransformation.build_wxo_run_body(
+ wxo_agent_id=wxo_agent_id, text=text, thread_id=thread_id
+ )
+
+ run_response = await client.post(
+ f"{base_url}/v1/orchestrate/runs",
+ json=body,
+ headers=auth_headers,
+ )
+ run_response.raise_for_status()
+ run_data: Dict[str, Any] = run_response.json()
+
+ status = run_data.get("status", "")
+ if status not in WatsonxOrchestrateTransformation.TERMINAL_STATES:
+ run_id = run_data.get("run_id") or run_data.get("id") or ""
+ if not run_id:
+ raise ValueError(f"WXO: No run_id in response: {run_data}")
+ run_data = await WatsonxOrchestrateHandler._poll_run(
+ base_url=base_url,
+ run_id=run_id,
+ auth_headers=auth_headers,
+ client=client,
+ )
+ status = run_data.get("status", "")
+
+ if status not in WatsonxOrchestrateTransformation.SUCCESS_STATES:
+ raise RuntimeError(
+ f"WXO run ended with non-success status '{status}': {run_data}"
+ )
+
+ response_text = WatsonxOrchestrateTransformation.extract_text_from_wxo_result(
+ run_data
+ )
+ return WatsonxOrchestrateTransformation.build_a2a_message_response(
+ request_id=request_id, text=response_text
+ )
+
+ @staticmethod
+ async def handle_streaming(
+ request_id: str,
+ params: Dict[str, Any],
+ litellm_params: Dict[str, Any],
+ chunk_size: int = 50,
+ delay_ms: int = 10,
+ ) -> AsyncIterator[Dict[str, Any]]:
+ (
+ cp4d_host,
+ instance_id,
+ wxo_agent_id,
+ api_key,
+ username,
+ auth_mode,
+ thread_id,
+ ) = WatsonxOrchestrateHandler._extract_litellm_params(litellm_params)
+
+ client = WatsonxOrchestrateHandler._http_client(timeout=120.0)
+ token = await WatsonxOrchestrateHandler._get_bearer_token(
+ cp4d_host=cp4d_host,
+ auth_mode=auth_mode,
+ api_key=api_key,
+ username=username,
+ client=client,
+ )
+ base_url = WatsonxOrchestrateTransformation.get_api_base_url(
+ cp4d_host, instance_id
+ )
+ auth_headers = {
+ "Authorization": f"Bearer {token}",
+ "Content-Type": "application/json",
+ }
+ text = WatsonxOrchestrateTransformation.extract_text_from_a2a_params(params)
+ body = WatsonxOrchestrateTransformation.build_wxo_run_body(
+ wxo_agent_id=wxo_agent_id, text=text, thread_id=thread_id
+ )
+
+ try:
+ response = await client.post(
+ f"{base_url}/v1/orchestrate/runs/stream",
+ json=body,
+ headers=auth_headers,
+ stream=True,
+ )
+ response.raise_for_status()
+ content_type = response.headers.get("content-type", "").lower()
+
+ if "text/event-stream" not in content_type:
+ response_body = await response.aread()
+ result = json.loads(response_body)
+ accumulated_text = (
+ WatsonxOrchestrateTransformation.extract_text_from_wxo_result(
+ result
+ )
+ )
+ else:
+ accumulated_text = (
+ await WatsonxOrchestrateHandler._accumulate_wxo_sse_text(response)
+ )
+
+ async for (
+ chunk
+ ) in WatsonxOrchestrateTransformation.fake_streaming_from_text(
+ text=accumulated_text,
+ request_id=request_id,
+ chunk_size=chunk_size,
+ delay_ms=delay_ms,
+ ):
+ yield chunk
+
+ except httpx.TransportError as exc:
+ verbose_logger.warning(
+ f"WXO: Streaming transport failed ({exc!r}), "
+ "falling back to non-streaming + fake streaming",
+ exc_info=True,
+ )
+ result = await WatsonxOrchestrateHandler.handle_non_streaming(
+ request_id=request_id,
+ params=params,
+ litellm_params=litellm_params,
+ )
+ response_text = (
+ WatsonxOrchestrateTransformation.extract_text_from_a2a_message_response(
+ result
+ )
+ )
+ async for (
+ chunk
+ ) in WatsonxOrchestrateTransformation.fake_streaming_from_text(
+ text=response_text,
+ request_id=request_id,
+ chunk_size=chunk_size,
+ delay_ms=delay_ms,
+ ):
+ yield chunk
diff --git a/litellm/a2a_protocol/providers/watsonx_orchestrate/transformation.py b/litellm/a2a_protocol/providers/watsonx_orchestrate/transformation.py
new file mode 100644
--- /dev/null
+++ b/litellm/a2a_protocol/providers/watsonx_orchestrate/transformation.py
@@ -1,0 +1,224 @@
+"""
+Transformation layer for IBM watsonx Orchestrate (WXO) agent provider.
+
+WXO uses a REST API (not A2A/JSON-RPC) with an async-poll execution model:
+ POST /v1/orchestrate/runs → submit run, get run_id
+ GET /v1/orchestrate/runs/{id} → poll until terminal state
+ POST /v1/orchestrate/runs/stream → native SSE streaming
+"""
+
+import asyncio
+from typing import Any, AsyncIterator, Dict, Optional
+from uuid import uuid4
+
+from litellm._logging import verbose_logger
+
+
+class WatsonxOrchestrateTransformation:
+ """
+ Handles request/response transformation between A2A and the WXO REST API.
+ """
+
+ TERMINAL_STATES = frozenset(
+ {"completed", "succeeded", "failed", "error", "cancelled"}
+ )
+ SUCCESS_STATES = frozenset({"completed", "succeeded"})
+
+ @staticmethod
+ def get_api_base_url(cp4d_host: str, instance_id: str) -> str:
+ """Build the WXO API base URL from host and instance ID."""
+ return f"{cp4d_host.rstrip('/')}/orchestrate/cpd/instances/{instance_id}"
+
+ @staticmethod
+ def extract_text_from_a2a_params(params: Dict[str, Any]) -> str:
+ """
+ Extract user message text from A2A MessageSendParams.
+
+ A2A format: params.message.parts[*] where part.kind == "text"
+ """
+ message = params.get("message", {})
+ parts = message.get("parts", [])
+ texts = []
+ for part in parts:
+ if not isinstance(part, dict):
+ continue
+ kind = part.get("kind")
+ if kind in (None, "", "text") and part.get("text"):
+ texts.append(part["text"])
+ return " ".join(texts) or ""
+
+ @staticmethod
+ def build_wxo_run_body(
+ wxo_agent_id: str,
+ text: str,
+ thread_id: Optional[str] = None,
+ ) -> Dict[str, Any]:
+ """Build the WXO POST /v1/orchestrate/runs request body."""
+ body: Dict[str, Any] = {
+ "agent_id": wxo_agent_id,
+ "message": {
+ "role": "user",
+ "content": [
+ {
+ "response_type": "conversational_search",
+ "text": text,
+ }
+ ],
+ },
+ }
+ if thread_id:
+ body["thread_id"] = thread_id
+ return body
+
+ @staticmethod
+ def extract_text_from_wxo_result(result: Any) -> str:
+ """
+ Extract response text from a WXO run result.
+
+ WXO can return text in several locations; checks in priority order per the API spec.
+ """
+ if not isinstance(result, dict):
+ return ""
+
+ # Primary: last_message.content[0].text
+ try:
+ text = result["last_message"]["content"][0]["text"]
+ if text:
+ return str(text)
+ except (KeyError, IndexError, TypeError):
+ pass
+
+ # Secondary: result.data.message.content[0].text
+ try:
+ text = result["result"]["data"]["message"]["content"][0]["text"]
+ if text:
+ return str(text)
+ except (KeyError, IndexError, TypeError):
+ pass
+
+ # Tertiary: results as a raw string
+ results = result.get("results")
+ if results and isinstance(results, str):
+ return results
+
+ return ""
+
+ @staticmethod
+ def extract_text_from_a2a_message_response(a2a_response: Dict[str, Any]) -> str:
+ result = a2a_response.get("result")
+ if not isinstance(result, dict):
+ verbose_logger.warning("WXO: A2A response missing result object")
+ return ""
+ parts = result.get("parts")
+ if not isinstance(parts, list):
+ verbose_logger.warning("WXO: A2A result has no parts list")
+ return ""
+ for part in parts:
+ if (
+ isinstance(part, dict)
+ and part.get("kind") == "text"
+ and part.get("text")
+ ):
+ return str(part["text"])
+ verbose_logger.warning("WXO: A2A result parts contained no text")
+ return ""
+
+ @staticmethod
+ def build_a2a_message_response(request_id: str, text: str) -> Dict[str, Any]:
+ """
+ Build a standard A2A non-streaming SendMessageResponse (kind=message).
+ """
+ return {
+ "jsonrpc": "2.0",
+ "id": request_id,
+ "result": {
+ "kind": "message",
+ "role": "agent",
+ "parts": [{"kind": "text", "text": text}],
+ "messageId": str(uuid4()),
+ },
+ }
+
+ @staticmethod
+ async def fake_streaming_from_text(
+ text: str,
+ request_id: str,
+ chunk_size: int = 50,
+ delay_ms: int = 10,
+ ) -> AsyncIterator[Dict[str, Any]]:
+ """
+ Emit standard A2A streaming events from a completed text response.
+
+ Event sequence:
+ 1. task (kind="task", state="submitted")
+ 2. status-update (kind="status-update", state="working")
+ 3. artifact-update chunks
+ 4. status-update (kind="status-update", state="completed", final=True)
+ """
+ task_id = str(uuid4())
+ context_id = str(uuid4())
+ artifact_id = str(uuid4())
+
+ # 1. Task submitted
+ yield {
+ "jsonrpc": "2.0",
+ "id": request_id,
+ "result": {
+ "contextId": context_id,
+ "id": task_id,
+ "kind": "task",
+ "status": {"state": "submitted"},
+ },
+ }
+
+ # 2. Working
+ yield {
+ "jsonrpc": "2.0",
+ "id": request_id,
+ "result": {
+ "contextId": context_id,
+ "final": False,
+ "kind": "status-update",
+ "status": {"state": "working"},
+ "taskId": task_id,
+ },
+ }
+ await asyncio.sleep(delay_ms / 1000.0)
+
+ # 3. Artifact chunks (always emit at least one chunk, even for empty text)
+ text_to_chunk = text or ""
+ for i in range(0, max(len(text_to_chunk), 1), chunk_size):
+ chunk_text = text_to_chunk[i : i + chunk_size]
+ is_last = (i + chunk_size) >= max(len(text_to_chunk), 1)
+ yield {
+ "jsonrpc": "2.0",
+ "id": request_id,
+ "result": {
+ "contextId": context_id,
+ "kind": "artifact-update",
+ "taskId": task_id,
+ "artifact": {
+ "artifactId": artifact_id,
+ "parts": [{"kind": "text", "text": chunk_text}],
+ },
+ },
+ }
+ if not is_last:
+ await asyncio.sleep(delay_ms / 1000.0)
+
+ # 4. Completed
+ yield {
+ "jsonrpc": "2.0",
+ "id": request_id,
+ "result": {
+ "contextId": context_id,
+ "final": True,
+ "kind": "status-update",
+ "status": {"state": "completed"},
+ "taskId": task_id,
+ },
+ }
+
+ verbose_logger.debug(
+ f"WXO: Fake streaming completed for request_id={request_id}"
+ )
diff --git a/litellm/proxy/public_endpoints/agent_create_fields.json b/litellm/proxy/public_endpoints/agent_create_fields.json
--- a/litellm/proxy/public_endpoints/agent_create_fields.json
+++ b/litellm/proxy/public_endpoints/agent_create_fields.json
@@ -189,6 +189,78 @@
"litellm_params_template": {
"custom_llm_provider": "vertex_ai"
}
+ },
+ {
+ "agent_type": "watsonx_orchestrate",
+ "agent_type_display_name": "watsonx Orchestrate",
+ "description": "Connect to IBM watsonx Orchestrate agents via CP4D or IBM Cloud IAM",
+ "logo_url": "/ui/assets/logos/watsonx.svg",
+ "credential_fields": [
+ {
+ "key": "cp4d_host",
+ "label": "CP4D Host URL",
+ "placeholder": "https://cpd-cpd.apps.example.com",
+ "tooltip": "Your CP4D cluster base URL (e.g. https://cpd-cpd.apps.example.com). For IBM Cloud WXO, use the service endpoint.",
+ "required": true,
+ "field_type": "text",
+ "default_value": null,
+ "include_in_litellm_params": true
+ },
+ {
+ "key": "instance_id",
+ "label": "WXO Instance ID",
+ "placeholder": "1769134113217795",
+ "tooltip": "The numeric watsonx Orchestrate instance ID. Find it in the WXO service URL: /orchestrate/cpd/instances/<INSTANCE_ID>",
+ "required": true,
+ "field_type": "text",
+ "default_value": null,
+ "include_in_litellm_params": true
+ },
+ {
+ "key": "wxo_agent_id",
+ "label": "WXO Agent ID",
+ "placeholder": "588c8cdf-60f4-454b-8468-8702b19dca46",
+ "tooltip": "UUID of the agent in watsonx Orchestrate. Find it via the WXO console or GET /v1/orchestrate/agents.",
+ "required": true,
+ "field_type": "text",
+ "default_value": null,
+ "include_in_litellm_params": true
+ },
+ {
+ "key": "auth_mode",
+ "label": "Authentication Mode",
+ "placeholder": null,
+ "tooltip": "cp4d: on-prem / CloudPak for Data (requires username). ibm_cloud: IBM Cloud IAM (api_key only).",
+ "required": false,
+ "field_type": "select",
+ "options": ["cp4d", "ibm_cloud"],
+ "default_value": "cp4d",
+ "include_in_litellm_params": true
+ },
+ {
+ "key": "username",
+ "label": "Username (CP4D only)",
+ "placeholder": "admin",
+ "tooltip": "Your CP4D username. Required when auth_mode is 'cp4d'.",
+ "required": false,
+ "field_type": "text",
+ "default_value": null,
+ "include_in_litellm_params": true
+ },
+ {
+ "key": "api_key",
+ "label": "API Key",
+ "placeholder": null,
+ "tooltip": "CP4D API key (auth_mode=cp4d) or IBM Cloud API key (auth_mode=ibm_cloud).",
+ "required": true,
+ "field_type": "password",
+ "default_value": null,
+ "include_in_litellm_params": true
+ }
+ ],
+ "litellm_params_template": {
+ "custom_llm_provider": "watsonx_orchestrate"
+ }
}
]
diff --git a/tests/test_litellm/a2a_protocol/providers/watsonx_orchestrate/test_watsonx_orchestrate_transformation.py b/tests/test_litellm/a2a_protocol/providers/watsonx_orchestrate/test_watsonx_orchestrate_transformation.py
new file mode 100644
--- /dev/null
+++ b/tests/test_litellm/a2a_protocol/providers/watsonx_orchestrate/test_watsonx_orchestrate_transformation.py
@@ -1,0 +1,265 @@
+import json
+import os
+import sys
+
+import pytest
+
+sys.path.insert(0, os.path.abspath("../../../../.."))
+
+from litellm.a2a_protocol.providers.config_manager import A2AProviderConfigManager
+from litellm.a2a_protocol.providers.watsonx_orchestrate.handler import (
+ WatsonxOrchestrateHandler,
+)
+from litellm.a2a_protocol.providers.watsonx_orchestrate.transformation import (
+ WatsonxOrchestrateTransformation,
+)
+
+
+class _JsonResponse:
+ def __init__(self, payload):
+ self.payload = payload
+
+ def raise_for_status(self):
+ pass
+
+ def json(self):
+ return self.payload
+
+
+class _ShortTtlTokenClient:
+ def __init__(self):
... diff truncated: showing 800 of 1035 linesYou can send follow-ups to the cloud agent here.
|
bugbot run |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.
Autofix Details
Bugbot Autofix prepared fixes for both issues found in the latest run.
- ✅ Fixed: Streaming non-SSE path skips status check and polling
- Added shared WXO run completion validation so non-SSE streaming JSON polls pending runs and raises on failed terminal states before extracting text.
- ✅ Fixed: Default auth mode requires username marked optional in dashboard
- Marked the dashboard username field required to match the default cp4d authentication mode runtime requirement.
Preview (4306dd1630)
diff --git a/litellm/a2a_protocol/providers/config_manager.py b/litellm/a2a_protocol/providers/config_manager.py
--- a/litellm/a2a_protocol/providers/config_manager.py
+++ b/litellm/a2a_protocol/providers/config_manager.py
@@ -48,4 +48,11 @@
return BedrockAgentCoreA2AConfig()
+ if custom_llm_provider == "watsonx_orchestrate":
+ from litellm.a2a_protocol.providers.watsonx_orchestrate.config import (
+ WatsonxOrchestrateA2AConfig,
+ )
+
+ return WatsonxOrchestrateA2AConfig()
+
return None
diff --git a/litellm/a2a_protocol/providers/watsonx_orchestrate/__init__.py b/litellm/a2a_protocol/providers/watsonx_orchestrate/__init__.py
new file mode 100644
--- /dev/null
+++ b/litellm/a2a_protocol/providers/watsonx_orchestrate/__init__.py
@@ -1,0 +1,3 @@
+"""
+IBM watsonx Orchestrate (WXO) A2A provider.
+"""
diff --git a/litellm/a2a_protocol/providers/watsonx_orchestrate/config.py b/litellm/a2a_protocol/providers/watsonx_orchestrate/config.py
new file mode 100644
--- /dev/null
+++ b/litellm/a2a_protocol/providers/watsonx_orchestrate/config.py
@@ -1,0 +1,55 @@
+"""
+A2A provider configuration for IBM watsonx Orchestrate (WXO).
+"""
+
+from typing import Any, AsyncIterator, Dict, Optional
+
+from litellm.a2a_protocol.providers.base import BaseA2AProviderConfig
+from litellm.a2a_protocol.providers.watsonx_orchestrate.handler import (
+ WatsonxOrchestrateHandler,
+)
+
+
+class WatsonxOrchestrateA2AConfig(BaseA2AProviderConfig):
+ """A2A bridge for IBM watsonx Orchestrate (REST runs API + poll/SSE)."""
+
+ async def handle_non_streaming(
+ self,
+ request_id: str,
+ params: Dict[str, Any],
+ api_base: Optional[str] = None,
+ **kwargs: Any,
+ ) -> Dict[str, Any]:
+ """Handle a non-streaming A2A request via WXO runs API."""
+ litellm_params = kwargs.get("litellm_params")
+ if not litellm_params:
+ raise ValueError(
+ "litellm_params is required for WatsonxOrchestrateA2AConfig "
+ "(must contain cp4d_host, instance_id, wxo_agent_id, api_key)"
+ )
+ return await WatsonxOrchestrateHandler.handle_non_streaming(
+ request_id=request_id,
+ params=params,
+ litellm_params=litellm_params,
+ )
+
+ async def handle_streaming(
+ self,
+ request_id: str,
+ params: Dict[str, Any],
+ api_base: Optional[str] = None,
+ **kwargs: Any,
+ ) -> AsyncIterator[Dict[str, Any]]:
+ """Handle a streaming A2A request via WXO streaming runs API."""
+ litellm_params = kwargs.get("litellm_params")
+ if not litellm_params:
+ raise ValueError(
+ "litellm_params is required for WatsonxOrchestrateA2AConfig "
+ "(must contain cp4d_host, instance_id, wxo_agent_id, api_key)"
+ )
+ async for chunk in WatsonxOrchestrateHandler.handle_streaming(
+ request_id=request_id,
+ params=params,
+ litellm_params=litellm_params,
+ ):
+ yield chunk
diff --git a/litellm/a2a_protocol/providers/watsonx_orchestrate/handler.py b/litellm/a2a_protocol/providers/watsonx_orchestrate/handler.py
new file mode 100644
--- /dev/null
+++ b/litellm/a2a_protocol/providers/watsonx_orchestrate/handler.py
@@ -1,0 +1,382 @@
+"""
+Handler for IBM watsonx Orchestrate (WXO) agent provider.
+"""
+
+import asyncio
+import hashlib
+import json
+import time
+from typing import Any, AsyncIterator, Dict, Optional, Tuple, cast
+
+import httpx
+
+from litellm._logging import verbose_logger
+from litellm.a2a_protocol.providers.watsonx_orchestrate.transformation import (
+ WatsonxOrchestrateTransformation,
+)
+from litellm.llms.custom_httpx.http_handler import (
+ AsyncHTTPHandler,
+ get_async_httpx_client,
+)
+from litellm.types.llms.custom_http import httpxSpecialProvider
+
+_IBM_CLOUD_IAM_URL = "https://iam.cloud.ibm.com/identity/token"
+_POLL_INTERVAL_S = 2.0
+_MAX_POLL_ATTEMPTS = 90
+_TOKEN_CACHE_TTL_BUFFER_S = 60
+_token_cache: Dict[str, Tuple[str, float]] = {}
+
+
+class WatsonxOrchestrateHandler:
+ @staticmethod
+ def _http_client(timeout: float = 90.0) -> AsyncHTTPHandler:
+ return get_async_httpx_client(
+ llm_provider=cast(Any, httpxSpecialProvider.A2AProvider),
+ params={"timeout": timeout},
+ )
+
+ @staticmethod
+ def _token_cache_key(
+ auth_mode: str,
+ cp4d_host: str,
+ api_key: str,
+ username: Optional[str],
+ ) -> str:
+ material = f"{auth_mode}:{cp4d_host}:{username or ''}:{api_key}"
+ return hashlib.sha256(material.encode()).hexdigest()
+
+ @staticmethod
+ def _cp4d_token_ttl_seconds(
+ expiration: Any, now_wall: Optional[float] = None
+ ) -> int:
+ # CP4D returns expiration as absolute Unix epoch seconds, not a duration.
+ expires_at = int(expiration)
+ wall = now_wall if now_wall is not None else time.time()
+ return max(expires_at - int(wall), 0)
+
+ @staticmethod
+ async def _get_bearer_token(
+ cp4d_host: str,
+ auth_mode: str,
+ api_key: str,
+ username: Optional[str] = None,
+ client: Optional[AsyncHTTPHandler] = None,
+ ) -> str:
+ cache_key = WatsonxOrchestrateHandler._token_cache_key(
+ auth_mode, cp4d_host, api_key, username
+ )
+ now = time.monotonic()
+ cached = _token_cache.get(cache_key)
+ if cached and cached[1] > now:
+ return cached[0]
+
+ if client is None:
+ client = WatsonxOrchestrateHandler._http_client(timeout=30.0)
+
+ if auth_mode == "ibm_cloud":
+ response = await client.post(
+ _IBM_CLOUD_IAM_URL,
+ data={
+ "grant_type": "urn:ibm:params:oauth:grant-type:apikey",
+ "apikey": api_key,
+ },
+ headers={"Content-Type": "application/x-www-form-urlencoded"},
+ )
+ response.raise_for_status()
+ payload = response.json()
+ token = str(payload["access_token"])
+ ttl_s = int(payload.get("expires_in", 3600))
+ else:
+ if not username:
+ raise ValueError(
+ "'username' is required in litellm_params when auth_mode='cp4d'"
+ )
+ token_url = f"{cp4d_host.rstrip('/')}/icp4d-api/v1/authorize"
+ response = await client.post(
+ token_url,
+ json={"username": username, "api_key": api_key},
+ headers={"Content-Type": "application/json"},
+ )
+ response.raise_for_status()
+ payload = response.json()
+ token = str(payload["token"])
+ expiration = payload.get("expiration")
+ if expiration is None:
+ ttl_s = 3600
+ else:
+ ttl_s = WatsonxOrchestrateHandler._cp4d_token_ttl_seconds(expiration)
+
+ expires_at = now + max(ttl_s - _TOKEN_CACHE_TTL_BUFFER_S, 0)
+ _token_cache[cache_key] = (token, expires_at)
+ return token
+
+ @staticmethod
+ async def _poll_run(
+ base_url: str,
+ run_id: str,
+ auth_headers: Dict[str, str],
+ client: AsyncHTTPHandler,
+ max_attempts: int = _MAX_POLL_ATTEMPTS,
+ interval_s: float = _POLL_INTERVAL_S,
+ ) -> Dict[str, Any]:
+ url = f"{base_url}/v1/orchestrate/runs/{run_id}"
+
+ for attempt in range(max_attempts):
+ await asyncio.sleep(interval_s)
+ response = await client.get(url, headers=auth_headers)
+ response.raise_for_status()
+ result: Dict[str, Any] = response.json()
+ status = result.get("status", "")
+ verbose_logger.debug(
+ f"WXO: Poll {attempt + 1}/{max_attempts} run='{run_id}' status='{status}'"
+ )
+ if status in WatsonxOrchestrateTransformation.TERMINAL_STATES:
+ return result
+
+ raise TimeoutError(
+ f"WXO run '{run_id}' did not reach a terminal state after "
+ f"{max_attempts * interval_s:.0f}s"
+ )
+
+ @staticmethod
+ async def _get_successful_run_data(
+ run_data: Dict[str, Any],
+ base_url: str,
+ auth_headers: Dict[str, str],
+ client: AsyncHTTPHandler,
+ ) -> Dict[str, Any]:
+ status = run_data.get("status", "")
+ if status not in WatsonxOrchestrateTransformation.TERMINAL_STATES:
+ run_id = run_data.get("run_id") or run_data.get("id") or ""
+ if not run_id:
+ raise ValueError(f"WXO: No run_id in response: {run_data}")
+ run_data = await WatsonxOrchestrateHandler._poll_run(
+ base_url=base_url,
+ run_id=run_id,
+ auth_headers=auth_headers,
+ client=client,
+ )
+ status = run_data.get("status", "")
+
+ if status not in WatsonxOrchestrateTransformation.SUCCESS_STATES:
+ raise RuntimeError(
+ f"WXO run ended with non-success status '{status}': {run_data}"
+ )
+
+ return run_data
+
+ @staticmethod
+ async def _accumulate_wxo_sse_text(response: Any) -> str:
+ accumulated_text = ""
+ async for line in response.aiter_lines():
+ if not line.startswith("data:"):
+ continue
+ data_str = line[5:].strip()
+ if not data_str or data_str == "[DONE]":
+ continue
+ try:
+ event = json.loads(data_str)
+ except json.JSONDecodeError:
+ continue
+ chunk_text = WatsonxOrchestrateTransformation.extract_text_from_wxo_result(
+ event
+ )
+ if chunk_text:
+ accumulated_text += chunk_text
+ return accumulated_text
+
+ @staticmethod
+ def _extract_litellm_params(litellm_params: Dict[str, Any]) -> tuple:
+ cp4d_host = litellm_params.get("cp4d_host") or ""
+ instance_id = litellm_params.get("instance_id") or ""
+ wxo_agent_id = litellm_params.get("wxo_agent_id") or ""
+ api_key = litellm_params.get("api_key") or ""
+ username = litellm_params.get("username") or None
+ auth_mode = litellm_params.get("auth_mode") or "cp4d"
+ thread_id = litellm_params.get("thread_id") or None
+
+ if not cp4d_host:
+ raise ValueError("'cp4d_host' is required in litellm_params for WXO agents")
+ if not instance_id:
+ raise ValueError(
+ "'instance_id' is required in litellm_params for WXO agents"
+ )
+ if not wxo_agent_id:
+ raise ValueError(
+ "'wxo_agent_id' is required in litellm_params for WXO agents"
+ )
+ if not api_key:
+ raise ValueError("'api_key' is required in litellm_params for WXO agents")
+
+ return (
+ cp4d_host,
+ instance_id,
+ wxo_agent_id,
+ api_key,
+ username,
+ auth_mode,
+ thread_id,
+ )
+
+ @staticmethod
+ async def handle_non_streaming(
+ request_id: str,
+ params: Dict[str, Any],
+ litellm_params: Dict[str, Any],
+ ) -> Dict[str, Any]:
+ (
+ cp4d_host,
+ instance_id,
+ wxo_agent_id,
+ api_key,
+ username,
+ auth_mode,
+ thread_id,
+ ) = WatsonxOrchestrateHandler._extract_litellm_params(litellm_params)
+
+ client = WatsonxOrchestrateHandler._http_client(timeout=90.0)
+ token = await WatsonxOrchestrateHandler._get_bearer_token(
+ cp4d_host=cp4d_host,
+ auth_mode=auth_mode,
+ api_key=api_key,
+ username=username,
+ client=client,
+ )
+ base_url = WatsonxOrchestrateTransformation.get_api_base_url(
+ cp4d_host, instance_id
+ )
+ auth_headers = {
+ "Authorization": f"Bearer {token}",
+ "Content-Type": "application/json",
+ "Accept": "application/json",
+ }
+
+ text = WatsonxOrchestrateTransformation.extract_text_from_a2a_params(params)
+ body = WatsonxOrchestrateTransformation.build_wxo_run_body(
+ wxo_agent_id=wxo_agent_id, text=text, thread_id=thread_id
+ )
+
+ run_response = await client.post(
+ f"{base_url}/v1/orchestrate/runs",
+ json=body,
+ headers=auth_headers,
+ )
+ run_response.raise_for_status()
+ run_data: Dict[str, Any] = run_response.json()
+
+ run_data = await WatsonxOrchestrateHandler._get_successful_run_data(
+ run_data=run_data,
+ base_url=base_url,
+ auth_headers=auth_headers,
+ client=client,
+ )
+
+ response_text = WatsonxOrchestrateTransformation.extract_text_from_wxo_result(
+ run_data
+ )
+ return WatsonxOrchestrateTransformation.build_a2a_message_response(
+ request_id=request_id, text=response_text
+ )
+
+ @staticmethod
+ async def handle_streaming(
+ request_id: str,
+ params: Dict[str, Any],
+ litellm_params: Dict[str, Any],
+ chunk_size: int = 50,
+ delay_ms: int = 10,
+ ) -> AsyncIterator[Dict[str, Any]]:
+ (
+ cp4d_host,
+ instance_id,
+ wxo_agent_id,
+ api_key,
+ username,
+ auth_mode,
+ thread_id,
+ ) = WatsonxOrchestrateHandler._extract_litellm_params(litellm_params)
+
+ client = WatsonxOrchestrateHandler._http_client(timeout=120.0)
+ token = await WatsonxOrchestrateHandler._get_bearer_token(
+ cp4d_host=cp4d_host,
+ auth_mode=auth_mode,
+ api_key=api_key,
+ username=username,
+ client=client,
+ )
+ base_url = WatsonxOrchestrateTransformation.get_api_base_url(
+ cp4d_host, instance_id
+ )
+ auth_headers = {
+ "Authorization": f"Bearer {token}",
+ "Content-Type": "application/json",
+ }
+ text = WatsonxOrchestrateTransformation.extract_text_from_a2a_params(params)
+ body = WatsonxOrchestrateTransformation.build_wxo_run_body(
+ wxo_agent_id=wxo_agent_id, text=text, thread_id=thread_id
+ )
+
+ try:
+ response = await client.post(
+ f"{base_url}/v1/orchestrate/runs/stream",
+ json=body,
+ headers=auth_headers,
+ stream=True,
+ )
+ response.raise_for_status()
+ content_type = response.headers.get("content-type", "").lower()
+
+ if "text/event-stream" not in content_type:
+ response_body = await response.aread()
+ result = json.loads(response_body)
+ result = await WatsonxOrchestrateHandler._get_successful_run_data(
+ run_data=result,
+ base_url=base_url,
+ auth_headers=auth_headers,
+ client=client,
+ )
+ accumulated_text = (
+ WatsonxOrchestrateTransformation.extract_text_from_wxo_result(
+ result
+ )
+ )
+ else:
+ accumulated_text = (
+ await WatsonxOrchestrateHandler._accumulate_wxo_sse_text(response)
+ )
+
+ async for (
+ chunk
+ ) in WatsonxOrchestrateTransformation.fake_streaming_from_text(
+ text=accumulated_text,
+ request_id=request_id,
+ chunk_size=chunk_size,
+ delay_ms=delay_ms,
+ ):
+ yield chunk
+
+ except httpx.TransportError as exc:
+ verbose_logger.warning(
+ f"WXO: Streaming transport failed ({exc!r}), "
+ "falling back to non-streaming + fake streaming",
+ exc_info=True,
+ )
+ result = await WatsonxOrchestrateHandler.handle_non_streaming(
+ request_id=request_id,
+ params=params,
+ litellm_params=litellm_params,
+ )
+ response_text = (
+ WatsonxOrchestrateTransformation.extract_text_from_a2a_message_response(
+ result
+ )
+ )
+ async for (
+ chunk
+ ) in WatsonxOrchestrateTransformation.fake_streaming_from_text(
+ text=response_text,
+ request_id=request_id,
+ chunk_size=chunk_size,
+ delay_ms=delay_ms,
+ ):
+ yield chunk
diff --git a/litellm/a2a_protocol/providers/watsonx_orchestrate/transformation.py b/litellm/a2a_protocol/providers/watsonx_orchestrate/transformation.py
new file mode 100644
--- /dev/null
+++ b/litellm/a2a_protocol/providers/watsonx_orchestrate/transformation.py
@@ -1,0 +1,224 @@
+"""
+Transformation layer for IBM watsonx Orchestrate (WXO) agent provider.
+
+WXO uses a REST API (not A2A/JSON-RPC) with an async-poll execution model:
+ POST /v1/orchestrate/runs → submit run, get run_id
+ GET /v1/orchestrate/runs/{id} → poll until terminal state
+ POST /v1/orchestrate/runs/stream → native SSE streaming
+"""
+
+import asyncio
+from typing import Any, AsyncIterator, Dict, Optional
+from uuid import uuid4
+
+from litellm._logging import verbose_logger
+
+
+class WatsonxOrchestrateTransformation:
+ """
+ Handles request/response transformation between A2A and the WXO REST API.
+ """
+
+ TERMINAL_STATES = frozenset(
+ {"completed", "succeeded", "failed", "error", "cancelled"}
+ )
+ SUCCESS_STATES = frozenset({"completed", "succeeded"})
+
+ @staticmethod
+ def get_api_base_url(cp4d_host: str, instance_id: str) -> str:
+ """Build the WXO API base URL from host and instance ID."""
+ return f"{cp4d_host.rstrip('/')}/orchestrate/cpd/instances/{instance_id}"
+
+ @staticmethod
+ def extract_text_from_a2a_params(params: Dict[str, Any]) -> str:
+ """
+ Extract user message text from A2A MessageSendParams.
+
+ A2A format: params.message.parts[*] where part.kind == "text"
+ """
+ message = params.get("message", {})
+ parts = message.get("parts", [])
+ texts = []
+ for part in parts:
+ if not isinstance(part, dict):
+ continue
+ kind = part.get("kind")
+ if kind in (None, "", "text") and part.get("text"):
+ texts.append(part["text"])
+ return " ".join(texts) or ""
+
+ @staticmethod
+ def build_wxo_run_body(
+ wxo_agent_id: str,
+ text: str,
+ thread_id: Optional[str] = None,
+ ) -> Dict[str, Any]:
+ """Build the WXO POST /v1/orchestrate/runs request body."""
+ body: Dict[str, Any] = {
+ "agent_id": wxo_agent_id,
+ "message": {
+ "role": "user",
+ "content": [
+ {
+ "response_type": "conversational_search",
+ "text": text,
+ }
+ ],
+ },
+ }
+ if thread_id:
+ body["thread_id"] = thread_id
+ return body
+
+ @staticmethod
+ def extract_text_from_wxo_result(result: Any) -> str:
+ """
+ Extract response text from a WXO run result.
+
+ WXO can return text in several locations; checks in priority order per the API spec.
+ """
+ if not isinstance(result, dict):
+ return ""
+
+ # Primary: last_message.content[0].text
+ try:
+ text = result["last_message"]["content"][0]["text"]
+ if text:
+ return str(text)
+ except (KeyError, IndexError, TypeError):
+ pass
+
+ # Secondary: result.data.message.content[0].text
+ try:
+ text = result["result"]["data"]["message"]["content"][0]["text"]
+ if text:
+ return str(text)
+ except (KeyError, IndexError, TypeError):
+ pass
+
+ # Tertiary: results as a raw string
+ results = result.get("results")
+ if results and isinstance(results, str):
+ return results
+
+ return ""
+
+ @staticmethod
+ def extract_text_from_a2a_message_response(a2a_response: Dict[str, Any]) -> str:
+ result = a2a_response.get("result")
+ if not isinstance(result, dict):
+ verbose_logger.warning("WXO: A2A response missing result object")
+ return ""
+ parts = result.get("parts")
+ if not isinstance(parts, list):
+ verbose_logger.warning("WXO: A2A result has no parts list")
+ return ""
+ for part in parts:
+ if (
+ isinstance(part, dict)
+ and part.get("kind") == "text"
+ and part.get("text")
+ ):
+ return str(part["text"])
+ verbose_logger.warning("WXO: A2A result parts contained no text")
+ return ""
+
+ @staticmethod
+ def build_a2a_message_response(request_id: str, text: str) -> Dict[str, Any]:
+ """
+ Build a standard A2A non-streaming SendMessageResponse (kind=message).
+ """
+ return {
+ "jsonrpc": "2.0",
+ "id": request_id,
+ "result": {
+ "kind": "message",
+ "role": "agent",
+ "parts": [{"kind": "text", "text": text}],
+ "messageId": str(uuid4()),
+ },
+ }
+
+ @staticmethod
+ async def fake_streaming_from_text(
+ text: str,
+ request_id: str,
+ chunk_size: int = 50,
+ delay_ms: int = 10,
+ ) -> AsyncIterator[Dict[str, Any]]:
+ """
+ Emit standard A2A streaming events from a completed text response.
+
+ Event sequence:
+ 1. task (kind="task", state="submitted")
+ 2. status-update (kind="status-update", state="working")
+ 3. artifact-update chunks
+ 4. status-update (kind="status-update", state="completed", final=True)
+ """
+ task_id = str(uuid4())
+ context_id = str(uuid4())
+ artifact_id = str(uuid4())
+
+ # 1. Task submitted
+ yield {
+ "jsonrpc": "2.0",
+ "id": request_id,
+ "result": {
+ "contextId": context_id,
+ "id": task_id,
+ "kind": "task",
+ "status": {"state": "submitted"},
+ },
+ }
+
+ # 2. Working
+ yield {
+ "jsonrpc": "2.0",
+ "id": request_id,
+ "result": {
+ "contextId": context_id,
+ "final": False,
+ "kind": "status-update",
+ "status": {"state": "working"},
+ "taskId": task_id,
+ },
+ }
+ await asyncio.sleep(delay_ms / 1000.0)
+
+ # 3. Artifact chunks (always emit at least one chunk, even for empty text)
+ text_to_chunk = text or ""
+ for i in range(0, max(len(text_to_chunk), 1), chunk_size):
+ chunk_text = text_to_chunk[i : i + chunk_size]
+ is_last = (i + chunk_size) >= max(len(text_to_chunk), 1)
+ yield {
+ "jsonrpc": "2.0",
+ "id": request_id,
+ "result": {
+ "contextId": context_id,
+ "kind": "artifact-update",
+ "taskId": task_id,
+ "artifact": {
+ "artifactId": artifact_id,
+ "parts": [{"kind": "text", "text": chunk_text}],
+ },
+ },
+ }
+ if not is_last:
+ await asyncio.sleep(delay_ms / 1000.0)
+
+ # 4. Completed
+ yield {
+ "jsonrpc": "2.0",
+ "id": request_id,
+ "result": {
+ "contextId": context_id,
+ "final": True,
+ "kind": "status-update",
+ "status": {"state": "completed"},
+ "taskId": task_id,
+ },
+ }
+
+ verbose_logger.debug(
+ f"WXO: Fake streaming completed for request_id={request_id}"
+ )
diff --git a/litellm/proxy/public_endpoints/agent_create_fields.json b/litellm/proxy/public_endpoints/agent_create_fields.json
--- a/litellm/proxy/public_endpoints/agent_create_fields.json
+++ b/litellm/proxy/public_endpoints/agent_create_fields.json
@@ -189,6 +189,78 @@
"litellm_params_template": {
"custom_llm_provider": "vertex_ai"
}
+ },
+ {
+ "agent_type": "watsonx_orchestrate",
+ "agent_type_display_name": "watsonx Orchestrate",
+ "description": "Connect to IBM watsonx Orchestrate agents via CP4D or IBM Cloud IAM",
+ "logo_url": "/ui/assets/logos/watsonx.svg",
+ "credential_fields": [
+ {
+ "key": "cp4d_host",
+ "label": "CP4D Host URL",
+ "placeholder": "https://cpd-cpd.apps.example.com",
+ "tooltip": "Your CP4D cluster base URL (e.g. https://cpd-cpd.apps.example.com). For IBM Cloud WXO, use the service endpoint.",
+ "required": true,
+ "field_type": "text",
+ "default_value": null,
+ "include_in_litellm_params": true
+ },
+ {
+ "key": "instance_id",
+ "label": "WXO Instance ID",
+ "placeholder": "1769134113217795",
+ "tooltip": "The numeric watsonx Orchestrate instance ID. Find it in the WXO service URL: /orchestrate/cpd/instances/<INSTANCE_ID>",
+ "required": true,
+ "field_type": "text",
+ "default_value": null,
+ "include_in_litellm_params": true
+ },
+ {
+ "key": "wxo_agent_id",
+ "label": "WXO Agent ID",
+ "placeholder": "588c8cdf-60f4-454b-8468-8702b19dca46",
+ "tooltip": "UUID of the agent in watsonx Orchestrate. Find it via the WXO console or GET /v1/orchestrate/agents.",
+ "required": true,
+ "field_type": "text",
+ "default_value": null,
+ "include_in_litellm_params": true
+ },
+ {
+ "key": "auth_mode",
+ "label": "Authentication Mode",
+ "placeholder": null,
+ "tooltip": "cp4d: on-prem / CloudPak for Data (requires username). ibm_cloud: IBM Cloud IAM (api_key only).",
+ "required": false,
+ "field_type": "select",
+ "options": ["cp4d", "ibm_cloud"],
+ "default_value": "cp4d",
+ "include_in_litellm_params": true
+ },
+ {
+ "key": "username",
+ "label": "Username (CP4D only)",
+ "placeholder": "admin",
+ "tooltip": "Your CP4D username. Required when auth_mode is 'cp4d'.",
+ "required": true,
+ "field_type": "text",
+ "default_value": null,
+ "include_in_litellm_params": true
+ },
+ {
+ "key": "api_key",
+ "label": "API Key",
+ "placeholder": null,
+ "tooltip": "CP4D API key (auth_mode=cp4d) or IBM Cloud API key (auth_mode=ibm_cloud).",
+ "required": true,
+ "field_type": "password",
+ "default_value": null,
+ "include_in_litellm_params": true
+ }
+ ],
+ "litellm_params_template": {
+ "custom_llm_provider": "watsonx_orchestrate"
+ }
}
]
diff --git a/tests/test_litellm/a2a_protocol/providers/watsonx_orchestrate/test_watsonx_orchestrate_transformation.py b/tests/test_litellm/a2a_protocol/providers/watsonx_orchestrate/test_watsonx_orchestrate_transformation.py
new file mode 100644
--- /dev/null
+++ b/tests/test_litellm/a2a_protocol/providers/watsonx_orchestrate/test_watsonx_orchestrate_transformation.py
@@ -1,0 +1,393 @@
+import json
+import os
+import sys
+from pathlib import Path
+
+import pytest
+
+sys.path.insert(0, os.path.abspath("../../../../.."))
... diff truncated: showing 800 of 1185 linesYou can send follow-ups to the cloud agent here.
|
bugbot run |
Username is only required for cp4d auth; ibm_cloud uses api_key alone. Backend still validates username when auth_mode is cp4d. Co-authored-by: Cursor <cursoragent@cursor.com>
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit e034cec. Configure here.
Username is not required in agent_create_fields.json; backend validates for cp4d auth_mode only. Co-authored-by: Cursor <cursoragent@cursor.com>
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 900926a. Configure here.
…itellm_watsonx_orchestrate_agents
Narrow the httpx.TransportError fallback in handle_streaming so it only covers the initial POST /runs/stream. Errors during polling or SSE consumption now propagate instead of triggering handle_non_streaming, which would have submitted a duplicate WXO run for the same request.
Return a typed WXORequestParams NamedTuple from _extract_litellm_params instead of a positional tuple so call sites read params by name, and send the user message with response_type 'text' so the run body is valid across all WXO agent configurations rather than the search-specific type.
…eoutError on poll timeout
* feat(a2a): add watsonx Orchestrate agent provider Bridge A2A message/send to WXO runs API (CP4D and IBM Cloud IAM auth), with dashboard agent type metadata and unit tests for transformations. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): use shared httpx client and cache WXO auth tokens Route WXO streaming through get_async_httpx_client (TLS verification enabled). Cache bearer tokens with TTL buffer. Extract A2A reply text via a dedicated helper instead of hard-coded JSON paths. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): treat CP4D token expiration as absolute Unix time CP4D /authorize returns expiration as epoch seconds, not TTL. Compute remaining lifetime against wall clock so cached tokens refresh before expiry. Co-authored-by: Cursor <cursoragent@cursor.com> * style(a2a): black-format watsonx orchestrate handler for CI py312 Co-authored-by: Cursor <cursoragent@cursor.com> * Fix watsonx orchestrate edge cases * Fix WXO streaming fallback error handling * Fix watsonx orchestrate run completion handling * fix(a2a): make WXO username optional in agent create UI Username is only required for cp4d auth; ibm_cloud uses api_key alone. Backend still validates username when auth_mode is cp4d. Co-authored-by: Cursor <cursoragent@cursor.com> * test(a2a): align WXO dashboard field test with optional username Username is not required in agent_create_fields.json; backend validates for cp4d auth_mode only. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): send Accept header on WXO streaming run request * fix(a2a/wxo): scope streaming transport fallback to initial POST only Narrow the httpx.TransportError fallback in handle_streaming so it only covers the initial POST /runs/stream. Errors during polling or SSE consumption now propagate instead of triggering handle_non_streaming, which would have submitted a duplicate WXO run for the same request. * refactor(a2a/wxo): type run-param extraction and use text response_type Return a typed WXORequestParams NamedTuple from _extract_litellm_params instead of a positional tuple so call sites read params by name, and send the user message with response_type 'text' so the run body is valid across all WXO agent configurations rather than the search-specific type. * fix(a2a/wxo): evict expired token cache entries and raise asyncio.TimeoutError on poll timeout --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
…9.0) (#93) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [ghcr.io/berriai/litellm](https://images.chainguard.dev/directory/image/wolfi-base/overview) ([source](https://github.com/BerriAI/litellm)) | minor | `v1.88.1` → `v1.89.0` | --- ### Release Notes <details> <summary>BerriAI/litellm (ghcr.io/berriai/litellm)</summary> ### [`v1.89.0`](https://github.com/BerriAI/litellm/releases/tag/v1.89.0) [Compare Source](https://github.com/BerriAI/litellm/compare/v1.89.0...v1.89.0) ##### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.89.0 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.89.0/cosign.pub \ ghcr.io/berriai/litellm:v1.89.0 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** ##### What's Changed - test(responses): bump deprecated gemini-3-pro-preview to gemini-3.1-pro-preview by [@​mateo-berri](https://github.com/mateo-berri) in [#​29433](https://github.com/BerriAI/litellm/pull/29433) - fix: map mistral/ministral-8b-latest in model price map by [@​mateo-berri](https://github.com/mateo-berri) in [#​29453](https://github.com/BerriAI/litellm/pull/29453) - fix(datadog): split oversized batches on 413 instead of re-queueing forever by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29444](https://github.com/BerriAI/litellm/pull/29444) - feat(otel): allowlist team\_metadata sub-keys promoted to baggage by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29442](https://github.com/BerriAI/litellm/pull/29442) - fix: stop use\_chat\_completions\_api flag from leaking into provider request body by [@​mateo-berri](https://github.com/mateo-berri) in [#​29447](https://github.com/BerriAI/litellm/pull/29447) - fix(anthropic, fireworks): inline legacy $ref defs in tool schemas by [@​milan-berri](https://github.com/milan-berri) in [#​28646](https://github.com/BerriAI/litellm/pull/28646) - fix(proxy): omit OpenAI \[DONE] on google-genai streamGenerateContent by [@​Sameerlite](https://github.com/Sameerlite) in [#​29426](https://github.com/BerriAI/litellm/pull/29426) - ci(release): create stable/X.Y.x line branch on X.Y.0 tags by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29457](https://github.com/BerriAI/litellm/pull/29457) - fix(vector-stores): support engines URL for Vertex AI Search by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​27885](https://github.com/BerriAI/litellm/pull/27885) - fix(ui): render caller-supplied filter options in caller order by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29462](https://github.com/BerriAI/litellm/pull/29462) - fix(batches): skip unnecessary batch input file reads by [@​Sameerlite](https://github.com/Sameerlite) in [#​29114](https://github.com/BerriAI/litellm/pull/29114) - docs(agents): clarify when to create new test files by [@​Sameerlite](https://github.com/Sameerlite) in [#​29472](https://github.com/BerriAI/litellm/pull/29472) - Litellm OSS Staging by [@​Sameerlite](https://github.com/Sameerlite) in [#​29161](https://github.com/BerriAI/litellm/pull/29161) - fix(mcp): clear allowed\_tools and tool overrides on MCP server edit by [@​Sameerlite](https://github.com/Sameerlite) in [#​29411](https://github.com/BerriAI/litellm/pull/29411) - Litellm OSS Staging 010626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​29422](https://github.com/BerriAI/litellm/pull/29422) - fix(ci): make CircleCI rerun-failed-tests collect tests when 2+ test files fail by [@​mateo-berri](https://github.com/mateo-berri) in [#​29475](https://github.com/BerriAI/litellm/pull/29475) - feat(a2a): watsonx Orchestrate agent provider by [@​Sameerlite](https://github.com/Sameerlite) in [#​29410](https://github.com/BerriAI/litellm/pull/29410) - fix(azure\_ai): strip tool-level extra fields on 400 and retry by [@​Sameerlite](https://github.com/Sameerlite) in [#​29479](https://github.com/BerriAI/litellm/pull/29479) - fix(docs): remove fixed dimensions from README hero image by [@​mateo-berri](https://github.com/mateo-berri) in [#​29496](https://github.com/BerriAI/litellm/pull/29496) - Litellm oss staging by [@​Sameerlite](https://github.com/Sameerlite) in [#​29492](https://github.com/BerriAI/litellm/pull/29492) - fix: small CLAUDE.md nits by [@​mateo-berri](https://github.com/mateo-berri) in [#​29504](https://github.com/BerriAI/litellm/pull/29504) - Add MCP semantic conventions to otelv2 by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29468](https://github.com/BerriAI/litellm/pull/29468) - fix(passthrough): emit otel guardrail span when a guardrail blocks by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29470](https://github.com/BerriAI/litellm/pull/29470) - fix(proxy): strip NUL bytes from spend log payloads to prevent PostgreSQL 22P05 by [@​milan-berri](https://github.com/milan-berri) in [#​29515](https://github.com/BerriAI/litellm/pull/29515) - \[internal copy of [#​28008](https://github.com/BerriAI/litellm/issues/28008)] Support MCP OAuth passthrough and issuer-scoped JWT auth by [@​mateo-berri](https://github.com/mateo-berri) in [#​28356](https://github.com/BerriAI/litellm/pull/28356) - feat(vector-stores): forward per-request params to Vertex AI Search by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29459](https://github.com/BerriAI/litellm/pull/29459) - feat(proxy): add per-MCP-server RPM rate limiting for keys and teams by [@​Sameerlite](https://github.com/Sameerlite) in [#​29482](https://github.com/BerriAI/litellm/pull/29482) - fix(tests): drop module-level test calls that break local\_testing collection by [@​mateo-berri](https://github.com/mateo-berri) in [#​29520](https://github.com/BerriAI/litellm/pull/29520) - feat(agents): add LangFlow agent provider with A2A session bridging by [@​Sameerlite](https://github.com/Sameerlite) in [#​28963](https://github.com/BerriAI/litellm/pull/28963) - fix(ui/agents): make A2A skill tags enterable and validated by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29512](https://github.com/BerriAI/litellm/pull/29512) - \[internal copy of [#​29232](https://github.com/BerriAI/litellm/issues/29232)] feat: route future Claude models to Anthropic provider via pattern matching by [@​mateo-berri](https://github.com/mateo-berri) in [#​29239](https://github.com/BerriAI/litellm/pull/29239) - fix(tests): drop import-time completion call in test\_register\_model by [@​mateo-berri](https://github.com/mateo-berri) in [#​29521](https://github.com/BerriAI/litellm/pull/29521) - test: stabilize batch VCR coverage and stop live upload/network leaks by [@​mateo-berri](https://github.com/mateo-berri) in [#​29477](https://github.com/BerriAI/litellm/pull/29477) - \[internal copy of [#​29003](https://github.com/BerriAI/litellm/issues/29003)] fix(vertex\_ai): use user-supplied api\_base as is for Model Garden OpenAI-compat path by [@​mateo-berri](https://github.com/mateo-berri) in [#​29530](https://github.com/BerriAI/litellm/pull/29530) - feat(proxy): native /health/drain preStop hook for graceful shutdown by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29439](https://github.com/BerriAI/litellm/pull/29439) - fix(auth): preserve 401 status for expired JWTs in OTel traces by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29510](https://github.com/BerriAI/litellm/pull/29510) - fix(otel): capture 401 error details in management endpoint spans by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29535](https://github.com/BerriAI/litellm/pull/29535) - test(proxy/utils): pin bottom-of-file helper behavior by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29509](https://github.com/BerriAI/litellm/pull/29509) - test(proxy/utils): pin PrismaClient and spend-update behavior by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29488](https://github.com/BerriAI/litellm/pull/29488) - test(proxy/utils): pin ProxyLogging behavior by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29485](https://github.com/BerriAI/litellm/pull/29485) - fix: missing span for guardrail passthrough by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29552](https://github.com/BerriAI/litellm/pull/29552) - fix(auth): let internal users view search tools by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29542](https://github.com/BerriAI/litellm/pull/29542) - fix: missing mcp otel attributes by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29554](https://github.com/BerriAI/litellm/pull/29554) - fix(proxy): resolve managed video model ids for auth by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​29545](https://github.com/BerriAI/litellm/pull/29545) - fix(key\_generate): allow team members to create keys on org-scoped teams by [@​milan-berri](https://github.com/milan-berri) in [#​29310](https://github.com/BerriAI/litellm/pull/29310) - test(pass-through): move Gemini pass-through tests to gemini-3.1-flash-lite by [@​mateo-berri](https://github.com/mateo-berri) in [#​29595](https://github.com/BerriAI/litellm/pull/29595) - Litellm oss staging 030626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​29578](https://github.com/BerriAI/litellm/pull/29578) - Fix : a2a bugs 030626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​29566](https://github.com/BerriAI/litellm/pull/29566) - \[internal copy of [#​29533](https://github.com/BerriAI/litellm/issues/29533)] fix(anthropic/adapter): emit thinking block for reasoning\_content-only streaming chunks by [@​mateo-berri](https://github.com/mateo-berri) in [#​29600](https://github.com/BerriAI/litellm/pull/29600) - ci: reproduce default-Windows wheel install to guard MAX\_PATH by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29597](https://github.com/BerriAI/litellm/pull/29597) - fix(vertex): strip output\_config.effort for Vertex Claude models that reject it (Haiku 4.5) by [@​mateo-berri](https://github.com/mateo-berri) in [#​29585](https://github.com/BerriAI/litellm/pull/29585) - Litellm websocket improvements by [@​Sameerlite](https://github.com/Sameerlite) in [#​29563](https://github.com/BerriAI/litellm/pull/29563) - feat(arize/phoenix): OpenInference rendering parity — tool\_calls, cost, passthrough I/O, session/user, multimodal, cache tokens by [@​milan-berri](https://github.com/milan-berri) in [#​28800](https://github.com/BerriAI/litellm/pull/28800) - \[internal copy of [#​29550](https://github.com/BerriAI/litellm/issues/29550)] fix: passthrough endpoints duplicate logs by [@​mateo-berri](https://github.com/mateo-berri) in [#​29598](https://github.com/BerriAI/litellm/pull/29598) - fix(ci): keep coverage rename green when a parallel node runs no tests by [@​mateo-berri](https://github.com/mateo-berri) in [#​29608](https://github.com/BerriAI/litellm/pull/29608) - test(vcr): close out the remaining VCR live-call leaks by [@​mateo-berri](https://github.com/mateo-berri) in [#​29603](https://github.com/BerriAI/litellm/pull/29603) - fix(key\_generate): exempt UI/CLI session tokens from the budget ceiling for team keys by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29612](https://github.com/BerriAI/litellm/pull/29612) - fix(realtime): allow null transcripts in stream logging payloads by [@​milan-berri](https://github.com/milan-berri) in [#​29625](https://github.com/BerriAI/litellm/pull/29625) - build(ui): migrate eslint to flat config + bump eslint-config-next to 16 by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29626](https://github.com/BerriAI/litellm/pull/29626) - fix(key\_generate): scope session-token team-key budget exemption to caller-supplied team\_id by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29641](https://github.com/BerriAI/litellm/pull/29641) - fix(proxy): disable proxy buffering on streaming SSE responses by [@​mateo-berri](https://github.com/mateo-berri) in [#​29557](https://github.com/BerriAI/litellm/pull/29557) - fix(mcp): gate /public/mcp\_hub strictly on litellm.public\_mcp\_servers by [@​michelligabriele](https://github.com/michelligabriele) in [#​27764](https://github.com/BerriAI/litellm/pull/27764) - ci(ui): frontend-lint job enforcing prettier + eslint on changed files by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29633](https://github.com/BerriAI/litellm/pull/29633) - fix(gemini): googleSearch + server-side tools and googleMaps JSON schema by [@​Sameerlite](https://github.com/Sameerlite) in [#​29582](https://github.com/BerriAI/litellm/pull/29582) - fix(proxy): passthrough 404 when SERVER\_ROOT\_PATH is set by [@​Sameerlite](https://github.com/Sameerlite) in [#​29658](https://github.com/BerriAI/litellm/pull/29658) - fix(gemini-realtime): use GA event names for Pipecat 1.3.x compatibility by [@​Sameerlite](https://github.com/Sameerlite) in [#​29662](https://github.com/BerriAI/litellm/pull/29662) - Litellm oss staging 040626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​29671](https://github.com/BerriAI/litellm/pull/29671) - style(ui): prettier formatting pass over the dashboard by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29622](https://github.com/BerriAI/litellm/pull/29622) - chore: ignore prettier dashboard reformat in git blame by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29695](https://github.com/BerriAI/litellm/pull/29695) - fix(helm): Enable Backend Deployment to mount Gateway config.yaml by [@​tin-berri](https://github.com/tin-berri) in [#​29605](https://github.com/BerriAI/litellm/pull/29605) - \[internal copy of [#​29277](https://github.com/BerriAI/litellm/issues/29277)] fix(proxy): add default=None to LiteLLM\_TeamMembership.litellm\_budget\_table by [@​mateo-berri](https://github.com/mateo-berri) in [#​29684](https://github.com/BerriAI/litellm/pull/29684) - test: make custom\_tokenizer proxy tests hermetic by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29643](https://github.com/BerriAI/litellm/pull/29643) - test(proxy): stop running real-DB tests in GitHub Actions unit jobs by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29700](https://github.com/BerriAI/litellm/pull/29700) - chore(ui): remove the bare-fetch lint rule by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29712](https://github.com/BerriAI/litellm/pull/29712) - Litellm jwt mapping virtualkeys by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​28510](https://github.com/BerriAI/litellm/pull/28510) - refactor(ui): shared HTTP client + location-pinned fetch() lint rule by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29723](https://github.com/BerriAI/litellm/pull/29723) - fix(proxy): stop team BYOK model name corruption on model edit by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29731](https://github.com/BerriAI/litellm/pull/29731) - \[internal copy of [#​29511](https://github.com/BerriAI/litellm/issues/29511)] feat(guardrails): add sensitive data routing to on-premise models by [@​mateo-berri](https://github.com/mateo-berri) in [#​29531](https://github.com/BerriAI/litellm/pull/29531) - fix(proxy/hooks): populate llm\_provider on internal rate-limit errors by [@​mateo-berri](https://github.com/mateo-berri) in [#​27707](https://github.com/BerriAI/litellm/pull/27707) - fix(vertex/anthropic): handle namespace tools and strip client\_metadata for codex compatibility by [@​Sameerlite](https://github.com/Sameerlite) in [#​29489](https://github.com/BerriAI/litellm/pull/29489) - Support OAuth M2M for Databricks Apps A2A agents by [@​mateo-berri](https://github.com/mateo-berri) in [#​29586](https://github.com/BerriAI/litellm/pull/29586) - fix: small CLAUDE.md nit by [@​mateo-berri](https://github.com/mateo-berri) in [#​29749](https://github.com/BerriAI/litellm/pull/29749) - fix(anthropic): route Claude Opus 4.8 through adaptive thinking by [@​mateo-berri](https://github.com/mateo-berri) in [#​29702](https://github.com/BerriAI/litellm/pull/29702) - fix(proxy): persist oauth2\_flow on MCP server registration by [@​michelligabriele](https://github.com/michelligabriele) in [#​29690](https://github.com/BerriAI/litellm/pull/29690) - \[internal copy of [#​27491](https://github.com/BerriAI/litellm/issues/27491)] fix(realtime): Fix Realtime Audio Token Cost Tracking by [@​mateo-berri](https://github.com/mateo-berri) in [#​29722](https://github.com/BerriAI/litellm/pull/29722) - fix(galileo): use ingest traces API and standard logging payload by [@​Sameerlite](https://github.com/Sameerlite) in [#​29651](https://github.com/BerriAI/litellm/pull/29651) - fix(auth): expand all-team-models sentinel in can\_key\_call\_model for batch validation by [@​Sameerlite](https://github.com/Sameerlite) in [#​29746](https://github.com/BerriAI/litellm/pull/29746) - test(vcr): stop refreshing cassette TTL on read so cassettes lapse after 24h by [@​mateo-berri](https://github.com/mateo-berri) in [#​29784](https://github.com/BerriAI/litellm/pull/29784) - test(ci): record/replay OpenAI image gen so the spend E2E isn't outage-bound by [@​mateo-berri](https://github.com/mateo-berri) in [#​29787](https://github.com/BerriAI/litellm/pull/29787) - fix(ui): route MCP playground auth by oauth2 mode instead of token\_url by [@​tin-berri](https://github.com/tin-berri) in [#​29714](https://github.com/BerriAI/litellm/pull/29714) - refactor(ui): centralize proxy base URL resolution into tested resolver by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29793](https://github.com/BerriAI/litellm/pull/29793) - Litellm oss staging 050626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​29774](https://github.com/BerriAI/litellm/pull/29774) - test(google): add google-genai SDK proxy integration tests by [@​Sameerlite](https://github.com/Sameerlite) in [#​29781](https://github.com/BerriAI/litellm/pull/29781) - fix(jwt): use resolved DB user\_id for spend on legacy email match by [@​milan-berri](https://github.com/milan-berri) in [#​29217](https://github.com/BerriAI/litellm/pull/29217) - feat(ui): generate dashboard API types from the proxy OpenAPI spec by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29816](https://github.com/BerriAI/litellm/pull/29816) - fix(proxy): drop deleted team BYOK model name from team.models by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29820](https://github.com/BerriAI/litellm/pull/29820) - feat(mcp): per-server env vars with global + per-user scopes by [@​mateo-berri](https://github.com/mateo-berri) in [#​28917](https://github.com/BerriAI/litellm/pull/28917) - refactor(ui): route behavior-preserving networking calls through apiClient by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29806](https://github.com/BerriAI/litellm/pull/29806) - fix(mcp): persist Tools-tab MCP OAuth token to DB by [@​tin-berri](https://github.com/tin-berri) in [#​29809](https://github.com/BerriAI/litellm/pull/29809) - fix(ui): require new expiration when regenerating an expired key by [@​milan-berri](https://github.com/milan-berri) in [#​29838](https://github.com/BerriAI/litellm/pull/29838) - refactor(ui): route query-building networking calls through apiClient by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29815](https://github.com/BerriAI/litellm/pull/29815) - Make the image-gen record/replay proxy report cache mode and per-request HIT/MISS by [@​mateo-berri](https://github.com/mateo-berri) in [#​29802](https://github.com/BerriAI/litellm/pull/29802) - feat(proxy): hot-reload .env in dev when running with --reload by [@​mateo-berri](https://github.com/mateo-berri) in [#​29783](https://github.com/BerriAI/litellm/pull/29783) - fix(ui): stop MCP playground tool calls from sending twice by [@​tin-berri](https://github.com/tin-berri) in [#​29821](https://github.com/BerriAI/litellm/pull/29821) - feat(fal\_ai): add Nano Banana / Gemini 2.5 Flash Image generation support by [@​mateo-berri](https://github.com/mateo-berri) in [#​29798](https://github.com/BerriAI/litellm/pull/29798) - Title: Fix managed batch cancel credential resolution by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​29734](https://github.com/BerriAI/litellm/pull/29734) - Title: fix(proxy): resolve vector store file list credentials from team deployments by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​29739](https://github.com/BerriAI/litellm/pull/29739) - refactor: convert AWS and GCP Terraform stacks into reusable modules … by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​28103](https://github.com/BerriAI/litellm/pull/28103) - chore(ui): build ui for release by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29853](https://github.com/BerriAI/litellm/pull/29853) - fix(terraform/gcp): prompt for image\_registry in DeployStack one-click by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29852](https://github.com/BerriAI/litellm/pull/29852) - fix(terraform/gcp): abandon SQL user on destroy by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29855](https://github.com/BerriAI/litellm/pull/29855) - Extend the record/replay proxy to chat, embeddings, moderations, rerank, and Anthropic by [@​mateo-berri](https://github.com/mateo-berri) in [#​29847](https://github.com/BerriAI/litellm/pull/29847) - chore(deps): bump deps by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29860](https://github.com/BerriAI/litellm/pull/29860) - chore(ci): promote internal staging to main by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29861](https://github.com/BerriAI/litellm/pull/29861) - fix: 400 on Anthropic context overflow; seed identity on failed auth by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29848](https://github.com/BerriAI/litellm/pull/29848) - chore(ci): promote internal staging to main by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29862](https://github.com/BerriAI/litellm/pull/29862) - chore(release): patch v1.89.0-rc.1 with [#​30064](https://github.com/BerriAI/litellm/issues/30064) (Claude Fable 5) for v1.89.0-rc.2 by [@​mateo-berri](https://github.com/mateo-berri) in [#​30143](https://github.com/BerriAI/litellm/pull/30143) **Full Changelog**: <https://github.com/BerriAI/litellm/compare/v1.88.0...v1.89.0> ### [`v1.89.0`](https://github.com/BerriAI/litellm/releases/tag/v1.89.0) [Compare Source](https://github.com/BerriAI/litellm/compare/v1.88.2...v1.89.0) ##### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.89.0 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.89.0/cosign.pub \ ghcr.io/berriai/litellm:v1.89.0 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** ##### What's Changed - test(responses): bump deprecated gemini-3-pro-preview to gemini-3.1-pro-preview by [@​mateo-berri](https://github.com/mateo-berri) in [#​29433](https://github.com/BerriAI/litellm/pull/29433) - fix: map mistral/ministral-8b-latest in model price map by [@​mateo-berri](https://github.com/mateo-berri) in [#​29453](https://github.com/BerriAI/litellm/pull/29453) - fix(datadog): split oversized batches on 413 instead of re-queueing forever by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29444](https://github.com/BerriAI/litellm/pull/29444) - feat(otel): allowlist team\_metadata sub-keys promoted to baggage by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29442](https://github.com/BerriAI/litellm/pull/29442) - fix: stop use\_chat\_completions\_api flag from leaking into provider request body by [@​mateo-berri](https://github.com/mateo-berri) in [#​29447](https://github.com/BerriAI/litellm/pull/29447) - fix(anthropic, fireworks): inline legacy $ref defs in tool schemas by [@​milan-berri](https://github.com/milan-berri) in [#​28646](https://github.com/BerriAI/litellm/pull/28646) - fix(proxy): omit OpenAI \[DONE] on google-genai streamGenerateContent by [@​Sameerlite](https://github.com/Sameerlite) in [#​29426](https://github.com/BerriAI/litellm/pull/29426) - ci(release): create stable/X.Y.x line branch on X.Y.0 tags by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29457](https://github.com/BerriAI/litellm/pull/29457) - fix(vector-stores): support engines URL for Vertex AI Search by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​27885](https://github.com/BerriAI/litellm/pull/27885) - fix(ui): render caller-supplied filter options in caller order by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29462](https://github.com/BerriAI/litellm/pull/29462) - fix(batches): skip unnecessary batch input file reads by [@​Sameerlite](https://github.com/Sameerlite) in [#​29114](https://github.com/BerriAI/litellm/pull/29114) - docs(agents): clarify when to create new test files by [@​Sameerlite](https://github.com/Sameerlite) in [#​29472](https://github.com/BerriAI/litellm/pull/29472) - Litellm OSS Staging by [@​Sameerlite](https://github.com/Sameerlite) in [#​29161](https://github.com/BerriAI/litellm/pull/29161) - fix(mcp): clear allowed\_tools and tool overrides on MCP server edit by [@​Sameerlite](https://github.com/Sameerlite) in [#​29411](https://github.com/BerriAI/litellm/pull/29411) - Litellm OSS Staging 010626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​29422](https://github.com/BerriAI/litellm/pull/29422) - fix(ci): make CircleCI rerun-failed-tests collect tests when 2+ test files fail by [@​mateo-berri](https://github.com/mateo-berri) in [#​29475](https://github.com/BerriAI/litellm/pull/29475) - feat(a2a): watsonx Orchestrate agent provider by [@​Sameerlite](https://github.com/Sameerlite) in [#​29410](https://github.com/BerriAI/litellm/pull/29410) - fix(azure\_ai): strip tool-level extra fields on 400 and retry by [@​Sameerlite](https://github.com/Sameerlite) in [#​29479](https://github.com/BerriAI/litellm/pull/29479) - fix(docs): remove fixed dimensions from README hero image by [@​mateo-berri](https://github.com/mateo-berri) in [#​29496](https://github.com/BerriAI/litellm/pull/29496) - Litellm oss staging by [@​Sameerlite](https://github.com/Sameerlite) in [#​29492](https://github.com/BerriAI/litellm/pull/29492) - fix: small CLAUDE.md nits by [@​mateo-berri](https://github.com/mateo-berri) in [#​29504](https://github.com/BerriAI/litellm/pull/29504) - Add MCP semantic conventions to otelv2 by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29468](https://github.com/BerriAI/litellm/pull/29468) - fix(passthrough): emit otel guardrail span when a guardrail blocks by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29470](https://github.com/BerriAI/litellm/pull/29470) - fix(proxy): strip NUL bytes from spend log payloads to prevent PostgreSQL 22P05 by [@​milan-berri](https://github.com/milan-berri) in [#​29515](https://github.com/BerriAI/litellm/pull/29515) - \[internal copy of [#​28008](https://github.com/BerriAI/litellm/issues/28008)] Support MCP OAuth passthrough and issuer-scoped JWT auth by [@​mateo-berri](https://github.com/mateo-berri) in [#​28356](https://github.com/BerriAI/litellm/pull/28356) - feat(vector-stores): forward per-request params to Vertex AI Search by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29459](https://github.com/BerriAI/litellm/pull/29459) - feat(proxy): add per-MCP-server RPM rate limiting for keys and teams by [@​Sameerlite](https://github.com/Sameerlite) in [#​29482](https://github.com/BerriAI/litellm/pull/29482) - fix(tests): drop module-level test calls that break local\_testing collection by [@​mateo-berri](https://github.com/mateo-berri) in [#​29520](https://github.com/BerriAI/litellm/pull/29520) - feat(agents): add LangFlow agent provider with A2A session bridging by [@​Sameerlite](https://github.com/Sameerlite) in [#​28963](https://github.com/BerriAI/litellm/pull/28963) - fix(ui/agents): make A2A skill tags enterable and validated by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29512](https://github.com/BerriAI/litellm/pull/29512) - \[internal copy of [#​29232](https://github.com/BerriAI/litellm/issues/29232)] feat: route future Claude models to Anthropic provider via pattern matching by [@​mateo-berri](https://github.com/mateo-berri) in [#​29239](https://github.com/BerriAI/litellm/pull/29239) - fix(tests): drop import-time completion call in test\_register\_model by [@​mateo-berri](https://github.com/mateo-berri) in [#​29521](https://github.com/BerriAI/litellm/pull/29521) - test: stabilize batch VCR coverage and stop live upload/network leaks by [@​mateo-berri](https://github.com/mateo-berri) in [#​29477](https://github.com/BerriAI/litellm/pull/29477) - \[internal copy of [#​29003](https://github.com/BerriAI/litellm/issues/29003)] fix(vertex\_ai): use user-supplied api\_base as is for Model Garden OpenAI-compat path by [@​mateo-berri](https://github.com/mateo-berri) in [#​29530](https://github.com/BerriAI/litellm/pull/29530) - feat(proxy): native /health/drain preStop hook for graceful shutdown by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29439](https://github.com/BerriAI/litellm/pull/29439) - fix(auth): preserve 401 status for expired JWTs in OTel traces by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29510](https://github.com/BerriAI/litellm/pull/29510) - fix(otel): capture 401 error details in management endpoint spans by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29535](https://github.com/BerriAI/litellm/pull/29535) - test(proxy/utils): pin bottom-of-file helper behavior by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29509](https://github.com/BerriAI/litellm/pull/29509) - test(proxy/utils): pin PrismaClient and spend-update behavior by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29488](https://github.com/BerriAI/litellm/pull/29488) - test(proxy/utils): pin ProxyLogging behavior by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29485](https://github.com/BerriAI/litellm/pull/29485) - fix: missing span for guardrail passthrough by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29552](https://github.com/BerriAI/litellm/pull/29552) - fix(auth): let internal users view search tools by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29542](https://github.com/BerriAI/litellm/pull/29542) - fix: missing mcp otel attributes by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29554](https://github.com/BerriAI/litellm/pull/29554) - fix(proxy): resolve managed video model ids for auth by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​29545](https://github.com/BerriAI/litellm/pull/29545) - fix(key\_generate): allow team members to create keys on org-scoped teams by [@​milan-berri](https://github.com/milan-berri) in [#​29310](https://github.com/BerriAI/litellm/pull/29310) - test(pass-through): move Gemini pass-through tests to gemini-3.1-flash-lite by [@​mateo-berri](https://github.com/mateo-berri) in [#​29595](https://github.com/BerriAI/litellm/pull/29595) - Litellm oss staging 030626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​29578](https://github.com/BerriAI/litellm/pull/29578) - Fix : a2a bugs 030626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​29566](https://github.com/BerriAI/litellm/pull/29566) - \[internal copy of [#​29533](https://github.com/BerriAI/litellm/issues/29533)] fix(anthropic/adapter): emit thinking block for reasoning\_content-only streaming chunks by [@​mateo-berri](https://github.com/mateo-berri) in [#​29600](https://github.com/BerriAI/litellm/pull/29600) - ci: reproduce default-Windows wheel install to guard MAX\_PATH by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29597](https://github.com/BerriAI/litellm/pull/29597) - fix(vertex): strip output\_config.effort for Vertex Claude models that reject it (Haiku 4.5) by [@​mateo-berri](https://github.com/mateo-berri) in [#​29585](https://github.com/BerriAI/litellm/pull/29585) - Litellm websocket improvements by [@​Sameerlite](https://github.com/Sameerlite) in [#​29563](https://github.com/BerriAI/litellm/pull/29563) - feat(arize/phoenix): OpenInference rendering parity — tool\_calls, cost, passthrough I/O, session/user, multimodal, cache tokens by [@​milan-berri](https://github.com/milan-berri) in [#​28800](https://github.com/BerriAI/litellm/pull/28800) - \[internal copy of [#​29550](https://github.com/BerriAI/litellm/issues/29550)] fix: passthrough endpoints duplicate logs by [@​mateo-berri](https://github.com/mateo-berri) in [#​29598](https://github.com/BerriAI/litellm/pull/29598) - fix(ci): keep coverage rename green when a parallel node runs no tests by [@​mateo-berri](https://github.com/mateo-berri) in [#​29608](https://github.com/BerriAI/litellm/pull/29608) - test(vcr): close out the remaining VCR live-call leaks by [@​mateo-berri](https://github.com/mateo-berri) in [#​29603](https://github.com/BerriAI/litellm/pull/29603) - fix(key\_generate): exempt UI/CLI session tokens from the budget ceiling for team keys by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29612](https://github.com/BerriAI/litellm/pull/29612) - fix(realtime): allow null transcripts in stream logging payloads by [@​milan-berri](https://github.com/milan-berri) in [#​29625](https://github.com/BerriAI/litellm/pull/29625) - build(ui): migrate eslint to flat config + bump eslint-config-next to 16 by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29626](https://github.com/BerriAI/litellm/pull/29626) - fix(key\_generate): scope session-token team-key budget exemption to caller-supplied team\_id by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29641](https://github.com/BerriAI/litellm/pull/29641) - fix(proxy): disable proxy buffering on streaming SSE responses by [@​mateo-berri](https://github.com/mateo-berri) in [#​29557](https://github.com/BerriAI/litellm/pull/29557) - fix(mcp): gate /public/mcp\_hub strictly on litellm.public\_mcp\_servers by [@​michelligabriele](https://github.com/michelligabriele) in [#​27764](https://github.com/BerriAI/litellm/pull/27764) - ci(ui): frontend-lint job enforcing prettier + eslint on changed files by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29633](https://github.com/BerriAI/litellm/pull/29633) - fix(gemini): googleSearch + server-side tools and googleMaps JSON schema by [@​Sameerlite](https://github.com/Sameerlite) in [#​29582](https://github.com/BerriAI/litellm/pull/29582) - fix(proxy): passthrough 404 when SERVER\_ROOT\_PATH is set by [@​Sameerlite](https://github.com/Sameerlite) in [#​29658](https://github.com/BerriAI/litellm/pull/29658) - fix(gemini-realtime): use GA event names for Pipecat 1.3.x compatibility by [@​Sameerlite](https://github.com/Sameerlite) in [#​29662](https://github.com/BerriAI/litellm/pull/29662) - Litellm oss staging 040626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​29671](https://github.com/BerriAI/litellm/pull/29671) - style(ui): prettier formatting pass over the dashboard by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29622](https://github.com/BerriAI/litellm/pull/29622) - chore: ignore prettier dashboard reformat in git blame by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29695](https://github.com/BerriAI/litellm/pull/29695) - fix(helm): Enable Backend Deployment to mount Gateway config.yaml by [@​tin-berri](https://github.com/tin-berri) in [#​29605](https://github.com/BerriAI/litellm/pull/29605) - \[internal copy of [#​29277](https://github.com/BerriAI/litellm/issues/29277)] fix(proxy): add default=None to LiteLLM\_TeamMembership.litellm\_budget\_table by [@​mateo-berri](https://github.com/mateo-berri) in [#​29684](https://github.com/BerriAI/litellm/pull/29684) - test: make custom\_tokenizer proxy tests hermetic by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29643](https://github.com/BerriAI/litellm/pull/29643) - test(proxy): stop running real-DB tests in GitHub Actions unit jobs by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29700](https://github.com/BerriAI/litellm/pull/29700) - chore(ui): remove the bare-fetch lint rule by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29712](https://github.com/BerriAI/litellm/pull/29712) - Litellm jwt mapping virtualkeys by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​28510](https://github.com/BerriAI/litellm/pull/28510) - refactor(ui): shared HTTP client + location-pinned fetch() lint rule by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29723](https://github.com/BerriAI/litellm/pull/29723) - fix(proxy): stop team BYOK model name corruption on model edit by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29731](https://github.com/BerriAI/litellm/pull/29731) - \[internal copy of [#​29511](https://github.com/BerriAI/litellm/issues/29511)] feat(guardrails): add sensitive data routing to on-premise models by [@​mateo-berri](https://github.com/mateo-berri) in [#​29531](https://github.com/BerriAI/litellm/pull/29531) - fix(proxy/hooks): populate llm\_provider on internal rate-limit errors by [@​mateo-berri](https://github.com/mateo-berri) in [#​27707](https://github.com/BerriAI/litellm/pull/27707) - fix(vertex/anthropic): handle namespace tools and strip client\_metadata for codex compatibility by [@​Sameerlite](https://github.com/Sameerlite) in [#​29489](https://github.com/BerriAI/litellm/pull/29489) - Support OAuth M2M for Databricks Apps A2A agents by [@​mateo-berri](https://github.com/mateo-berri) in [#​29586](https://github.com/BerriAI/litellm/pull/29586) - fix: small CLAUDE.md nit by [@​mateo-berri](https://github.com/mateo-berri) in [#​29749](https://github.com/BerriAI/litellm/pull/29749) - fix(anthropic): route Claude Opus 4.8 through adaptive thinking by [@​mateo-berri](https://github.com/mateo-berri) in [#​29702](https://github.com/BerriAI/litellm/pull/29702) - fix(proxy): persist oauth2\_flow on MCP server registration by [@​michelligabriele](https://github.com/michelligabriele) in [#​29690](https://github.com/BerriAI/litellm/pull/29690) - \[internal copy of [#​27491](https://github.com/BerriAI/litellm/issues/27491)] fix(realtime): Fix Realtime Audio Token Cost Tracking by [@​mateo-berri](https://github.com/mateo-berri) in [#​29722](https://github.com/BerriAI/litellm/pull/29722) - fix(galileo): use ingest traces API and standard logging payload by [@​Sameerlite](https://github.com/Sameerlite) in [#​29651](https://github.com/BerriAI/litellm/pull/29651) - fix(auth): expand all-team-models sentinel in can\_key\_call\_model for batch validation by [@​Sameerlite](https://github.com/Sameerlite) in [#​29746](https://github.com/BerriAI/litellm/pull/29746) - test(vcr): stop refreshing cassette TTL on read so cassettes lapse after 24h by [@​mateo-berri](https://github.com/mateo-berri) in [#​29784](https://github.com/BerriAI/litellm/pull/29784) - test(ci): record/replay OpenAI image gen so the spend E2E isn't outage-bound by [@​mateo-berri](https://github.com/mateo-berri) in [#​29787](https://github.com/BerriAI/litellm/pull/29787) - fix(ui): route MCP playground auth by oauth2 mode instead of token\_url by [@​tin-berri](https://github.com/tin-berri) in [#​29714](https://github.com/BerriAI/litellm/pull/29714) - refactor(ui): centralize proxy base URL resolution into tested resolver by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29793](https://github.com/BerriAI/litellm/pull/29793) - Litellm oss staging 050626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​29774](https://github.com/BerriAI/litellm/pull/29774) - test(google): add google-genai SDK proxy integration tests by [@​Sameerlite](https://github.com/Sameerlite) in [#​29781](https://github.com/BerriAI/litellm/pull/29781) - fix(jwt): use resolved DB user\_id for spend on legacy email match by [@​milan-berri](https://github.com/milan-berri) in [#​29217](https://github.com/BerriAI/litellm/pull/29217) - feat(ui): generate dashboard API types from the proxy OpenAPI spec by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29816](https://github.com/BerriAI/litellm/pull/29816) - fix(proxy): drop deleted team BYOK model name from team.models by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29820](https://github.com/BerriAI/litellm/pull/29820) - feat(mcp): per-server env vars with global + per-user scopes by [@​mateo-berri](https://github.com/mateo-berri) in [#​28917](https://github.com/BerriAI/litellm/pull/28917) - refactor(ui): route behavior-preserving networking calls through apiClient by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29806](https://github.com/BerriAI/litellm/pull/29806) - fix(mcp): persist Tools-tab MCP OAuth token to DB by [@​tin-berri](https://github.com/tin-berri) in [#​29809](https://github.com/BerriAI/litellm/pull/29809) - fix(ui): require new expiration when regenerating an expired key by [@​milan-berri](https://github.com/milan-berri) in [#​29838](https://github.com/BerriAI/litellm/pull/29838) - refactor(ui): route query-building networking calls through apiClient by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29815](https://github.com/BerriAI/litellm/pull/29815) - Make the image-gen record/replay proxy report cache mode and per-request HIT/MISS by [@​mateo-berri](https://github.com/mateo-berri) in [#​29802](https://github.com/BerriAI/litellm/pull/29802) - feat(proxy): hot-reload .env in dev when running with --reload by [@​mateo-berri](https://github.com/mateo-berri) in [#​29783](https://github.com/BerriAI/litellm/pull/29783) - fix(ui): stop MCP playground tool calls from sending twice by [@​tin-berri](https://github.com/tin-berri) in [#​29821](https://github.com/BerriAI/litellm/pull/29821) - feat(fal\_ai): add Nano Banana / Gemini 2.5 Flash Image generation support by [@​mateo-berri](https://github.com/mateo-berri) in [#​29798](https://github.com/BerriAI/litellm/pull/29798) - Title: Fix managed batch cancel credential resolution by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​29734](https://github.com/BerriAI/litellm/pull/29734) - Title: fix(proxy): resolve vector store file list credentials from team deployments by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​29739](https://github.com/BerriAI/litellm/pull/29739) - refactor: convert AWS and GCP Terraform stacks into reusable modules … by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​28103](https://github.com/BerriAI/litellm/pull/28103) - chore(ui): build ui for release by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29853](https://github.com/BerriAI/litellm/pull/29853) - fix(terraform/gcp): prompt for image\_registry in DeployStack one-click by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29852](https://github.com/BerriAI/litellm/pull/29852) - fix(terraform/gcp): abandon SQL user on destroy by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29855](https://github.com/BerriAI/litellm/pull/29855) - Extend the record/replay proxy to chat, embeddings, moderations, rerank, and Anthropic by [@​mateo-berri](https://github.com/mateo-berri) in [#​29847](https://github.com/BerriAI/litellm/pull/29847) - chore(deps): bump deps by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29860](https://github.com/BerriAI/litellm/pull/29860) - chore(ci): promote internal staging to main by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29861](https://github.com/BerriAI/litellm/pull/29861) - fix: 400 on Anthropic context overflow; seed identity on failed auth by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29848](https://github.com/BerriAI/litellm/pull/29848) - chore(ci): promote internal staging to main by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29862](https://github.com/BerriAI/litellm/pull/29862) - chore(release): patch v1.89.0-rc.1 with [#​30064](https://github.com/BerriAI/litellm/issues/30064) (Claude Fable 5) for v1.89.0-rc.2 by [@​mateo-berri](https://github.com/mateo-berri) in [#​30143](https://github.com/BerriAI/litellm/pull/30143) **Full Changelog**: <https://github.com/BerriAI/litellm/compare/v1.88.0...v1.89.0> ### [`v1.88.2`](https://github.com/BerriAI/litellm/releases/tag/v1.88.2) [Compare Source](https://github.com/BerriAI/litellm/compare/v1.88.2...v1.88.2) ##### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.88.2 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.88.2/cosign.pub \ ghcr.io/berriai/litellm:v1.88.2 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** ##### What's Changed - chore(release): backport Fable 5, batch-file auth, CrowdStrike AIDR, Mantle Responses SigV4, and NetApp streaming-cost fix to stable/1.88.x and cut 1.88.2 by [@​mateo-berri](https://github.com/mateo-berri) in [#​30144](https://github.com/BerriAI/litellm/pull/30144) - chore(release): backport DB-resilience, passthrough, model-info, budget, and deps fixes to stable/1.88.x by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30408](https://github.com/BerriAI/litellm/pull/30408) **Full Changelog**: <https://github.com/BerriAI/litellm/compare/v1.88.1...v1.88.2> ### [`v1.88.2`](https://github.com/BerriAI/litellm/releases/tag/v1.88.2) [Compare Source](https://github.com/BerriAI/litellm/compare/v1.88.1...v1.88.2) ##### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.88.2 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.88.2/cosign.pub \ ghcr.io/berriai/litellm:v1.88.2 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** ##### What's Changed - chore(release): backport Fable 5, batch-file auth, CrowdStrike AIDR, Mantle Responses SigV4, and NetApp streaming-cost fix to stable/1.88.x and cut 1.88.2 by [@​mateo-berri](https://github.com/mateo-berri) in [#​30144](https://github.com/BerriAI/litellm/pull/30144) - chore(release): backport DB-resilience, passthrough, model-info, budget, and deps fixes to stable/1.88.x by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30408](https://github.com/BerriAI/litellm/pull/30408) **Full Changelog**: <https://github.com/BerriAI/litellm/compare/v1.88.1...v1.88.2> </details> --- ### Configuration 📅 **Schedule**: (in timezone Europe/London) - Branch creation - At any time (no schedule defined) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about these updates again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4yMTkuMCIsInVwZGF0ZWRJblZlciI6IjQzLjIxOS4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJyZW5vdmF0ZS9jb250YWluZXIiLCJ0eXBlL21pbm9yIl19--> Reviewed-on: https://forgejo.hayden.moe/hayden/phoebe/pulls/93
…to v1.89.0 (#200)
This PR contains the following updates:
| Package | Update | Change |
|---|---|---|
| [https://github.com/BerriAI/litellm.git](https://github.com/BerriAI/litellm) | minor | `v1.85.1` → `v1.89.0` |
---
> ⚠️ **Warning**
>
> Some dependencies could not be looked up. Check the [Dependency Dashboard](issues/155) for more information.
---
### Release Notes
<details>
<summary>BerriAI/litellm (https://github.com/BerriAI/litellm.git)</summary>
### [`v1.89.0`](https://github.com/BerriAI/litellm/releases/tag/v1.89.0)
[Compare Source](https://github.com/BerriAI/litellm/compare/v1.88.2...v1.89.0)
#### Verify Docker Image Signature
All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0).
**Verify using the pinned commit hash (recommended):**
A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:
```bash
cosign verify \
--key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
ghcr.io/berriai/litellm:v1.89.0
```
**Verify using the release tag (convenience):**
Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:
```bash
cosign verify \
--key https://raw.githubusercontent.com/BerriAI/litellm/v1.89.0/cosign.pub \
ghcr.io/berriai/litellm:v1.89.0
```
Expected output:
```
The following checks were performed on each of these signatures:
- The cosign claims were validated
- The signatures were verified against the specified public key
```
***
#### What's Changed
- test(responses): bump deprecated gemini-3-pro-preview to gemini-3.1-pro-preview by [@​mateo-berri](https://github.com/mateo-berri) in [#​29433](https://github.com/BerriAI/litellm/pull/29433)
- fix: map mistral/ministral-8b-latest in model price map by [@​mateo-berri](https://github.com/mateo-berri) in [#​29453](https://github.com/BerriAI/litellm/pull/29453)
- fix(datadog): split oversized batches on 413 instead of re-queueing forever by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29444](https://github.com/BerriAI/litellm/pull/29444)
- feat(otel): allowlist team\_metadata sub-keys promoted to baggage by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29442](https://github.com/BerriAI/litellm/pull/29442)
- fix: stop use\_chat\_completions\_api flag from leaking into provider request body by [@​mateo-berri](https://github.com/mateo-berri) in [#​29447](https://github.com/BerriAI/litellm/pull/29447)
- fix(anthropic, fireworks): inline legacy $ref defs in tool schemas by [@​milan-berri](https://github.com/milan-berri) in [#​28646](https://github.com/BerriAI/litellm/pull/28646)
- fix(proxy): omit OpenAI \[DONE] on google-genai streamGenerateContent by [@​Sameerlite](https://github.com/Sameerlite) in [#​29426](https://github.com/BerriAI/litellm/pull/29426)
- ci(release): create stable/X.Y.x line branch on X.Y.0 tags by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29457](https://github.com/BerriAI/litellm/pull/29457)
- fix(vector-stores): support engines URL for Vertex AI Search by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​27885](https://github.com/BerriAI/litellm/pull/27885)
- fix(ui): render caller-supplied filter options in caller order by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29462](https://github.com/BerriAI/litellm/pull/29462)
- fix(batches): skip unnecessary batch input file reads by [@​Sameerlite](https://github.com/Sameerlite) in [#​29114](https://github.com/BerriAI/litellm/pull/29114)
- docs(agents): clarify when to create new test files by [@​Sameerlite](https://github.com/Sameerlite) in [#​29472](https://github.com/BerriAI/litellm/pull/29472)
- Litellm OSS Staging by [@​Sameerlite](https://github.com/Sameerlite) in [#​29161](https://github.com/BerriAI/litellm/pull/29161)
- fix(mcp): clear allowed\_tools and tool overrides on MCP server edit by [@​Sameerlite](https://github.com/Sameerlite) in [#​29411](https://github.com/BerriAI/litellm/pull/29411)
- Litellm OSS Staging 010626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​29422](https://github.com/BerriAI/litellm/pull/29422)
- fix(ci): make CircleCI rerun-failed-tests collect tests when 2+ test files fail by [@​mateo-berri](https://github.com/mateo-berri) in [#​29475](https://github.com/BerriAI/litellm/pull/29475)
- feat(a2a): watsonx Orchestrate agent provider by [@​Sameerlite](https://github.com/Sameerlite) in [#​29410](https://github.com/BerriAI/litellm/pull/29410)
- fix(azure\_ai): strip tool-level extra fields on 400 and retry by [@​Sameerlite](https://github.com/Sameerlite) in [#​29479](https://github.com/BerriAI/litellm/pull/29479)
- fix(docs): remove fixed dimensions from README hero image by [@​mateo-berri](https://github.com/mateo-berri) in [#​29496](https://github.com/BerriAI/litellm/pull/29496)
- Litellm oss staging by [@​Sameerlite](https://github.com/Sameerlite) in [#​29492](https://github.com/BerriAI/litellm/pull/29492)
- fix: small CLAUDE.md nits by [@​mateo-berri](https://github.com/mateo-berri) in [#​29504](https://github.com/BerriAI/litellm/pull/29504)
- Add MCP semantic conventions to otelv2 by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29468](https://github.com/BerriAI/litellm/pull/29468)
- fix(passthrough): emit otel guardrail span when a guardrail blocks by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29470](https://github.com/BerriAI/litellm/pull/29470)
- fix(proxy): strip NUL bytes from spend log payloads to prevent PostgreSQL 22P05 by [@​milan-berri](https://github.com/milan-berri) in [#​29515](https://github.com/BerriAI/litellm/pull/29515)
- \[internal copy of [#​28008](https://github.com/BerriAI/litellm/issues/28008)] Support MCP OAuth passthrough and issuer-scoped JWT auth by [@​mateo-berri](https://github.com/mateo-berri) in [#​28356](https://github.com/BerriAI/litellm/pull/28356)
- feat(vector-stores): forward per-request params to Vertex AI Search by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29459](https://github.com/BerriAI/litellm/pull/29459)
- feat(proxy): add per-MCP-server RPM rate limiting for keys and teams by [@​Sameerlite](https://github.com/Sameerlite) in [#​29482](https://github.com/BerriAI/litellm/pull/29482)
- fix(tests): drop module-level test calls that break local\_testing collection by [@​mateo-berri](https://github.com/mateo-berri) in [#​29520](https://github.com/BerriAI/litellm/pull/29520)
- feat(agents): add LangFlow agent provider with A2A session bridging by [@​Sameerlite](https://github.com/Sameerlite) in [#​28963](https://github.com/BerriAI/litellm/pull/28963)
- fix(ui/agents): make A2A skill tags enterable and validated by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29512](https://github.com/BerriAI/litellm/pull/29512)
- \[internal copy of [#​29232](https://github.com/BerriAI/litellm/issues/29232)] feat: route future Claude models to Anthropic provider via pattern matching by [@​mateo-berri](https://github.com/mateo-berri) in [#​29239](https://github.com/BerriAI/litellm/pull/29239)
- fix(tests): drop import-time completion call in test\_register\_model by [@​mateo-berri](https://github.com/mateo-berri) in [#​29521](https://github.com/BerriAI/litellm/pull/29521)
- test: stabilize batch VCR coverage and stop live upload/network leaks by [@​mateo-berri](https://github.com/mateo-berri) in [#​29477](https://github.com/BerriAI/litellm/pull/29477)
- \[internal copy of [#​29003](https://github.com/BerriAI/litellm/issues/29003)] fix(vertex\_ai): use user-supplied api\_base as is for Model Garden OpenAI-compat path by [@​mateo-berri](https://github.com/mateo-berri) in [#​29530](https://github.com/BerriAI/litellm/pull/29530)
- feat(proxy): native /health/drain preStop hook for graceful shutdown by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29439](https://github.com/BerriAI/litellm/pull/29439)
- fix(auth): preserve 401 status for expired JWTs in OTel traces by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29510](https://github.com/BerriAI/litellm/pull/29510)
- fix(otel): capture 401 error details in management endpoint spans by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29535](https://github.com/BerriAI/litellm/pull/29535)
- test(proxy/utils): pin bottom-of-file helper behavior by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29509](https://github.com/BerriAI/litellm/pull/29509)
- test(proxy/utils): pin PrismaClient and spend-update behavior by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29488](https://github.com/BerriAI/litellm/pull/29488)
- test(proxy/utils): pin ProxyLogging behavior by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29485](https://github.com/BerriAI/litellm/pull/29485)
- fix: missing span for guardrail passthrough by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29552](https://github.com/BerriAI/litellm/pull/29552)
- fix(auth): let internal users view search tools by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29542](https://github.com/BerriAI/litellm/pull/29542)
- fix: missing mcp otel attributes by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29554](https://github.com/BerriAI/litellm/pull/29554)
- fix(proxy): resolve managed video model ids for auth by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​29545](https://github.com/BerriAI/litellm/pull/29545)
- fix(key\_generate): allow team members to create keys on org-scoped teams by [@​milan-berri](https://github.com/milan-berri) in [#​29310](https://github.com/BerriAI/litellm/pull/29310)
- test(pass-through): move Gemini pass-through tests to gemini-3.1-flash-lite by [@​mateo-berri](https://github.com/mateo-berri) in [#​29595](https://github.com/BerriAI/litellm/pull/29595)
- Litellm oss staging 030626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​29578](https://github.com/BerriAI/litellm/pull/29578)
- Fix : a2a bugs 030626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​29566](https://github.com/BerriAI/litellm/pull/29566)
- \[internal copy of [#​29533](https://github.com/BerriAI/litellm/issues/29533)] fix(anthropic/adapter): emit thinking block for reasoning\_content-only streaming chunks by [@​mateo-berri](https://github.com/mateo-berri) in [#​29600](https://github.com/BerriAI/litellm/pull/29600)
- ci: reproduce default-Windows wheel install to guard MAX\_PATH by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29597](https://github.com/BerriAI/litellm/pull/29597)
- fix(vertex): strip output\_config.effort for Vertex Claude models that reject it (Haiku 4.5) by [@​mateo-berri](https://github.com/mateo-berri) in [#​29585](https://github.com/BerriAI/litellm/pull/29585)
- Litellm websocket improvements by [@​Sameerlite](https://github.com/Sameerlite) in [#​29563](https://github.com/BerriAI/litellm/pull/29563)
- feat(arize/phoenix): OpenInference rendering parity — tool\_calls, cost, passthrough I/O, session/user, multimodal, cache tokens by [@​milan-berri](https://github.com/milan-berri) in [#​28800](https://github.com/BerriAI/litellm/pull/28800)
- \[internal copy of [#​29550](https://github.com/BerriAI/litellm/issues/29550)] fix: passthrough endpoints duplicate logs by [@​mateo-berri](https://github.com/mateo-berri) in [#​29598](https://github.com/BerriAI/litellm/pull/29598)
- fix(ci): keep coverage rename green when a parallel node runs no tests by [@​mateo-berri](https://github.com/mateo-berri) in [#​29608](https://github.com/BerriAI/litellm/pull/29608)
- test(vcr): close out the remaining VCR live-call leaks by [@​mateo-berri](https://github.com/mateo-berri) in [#​29603](https://github.com/BerriAI/litellm/pull/29603)
- fix(key\_generate): exempt UI/CLI session tokens from the budget ceiling for team keys by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29612](https://github.com/BerriAI/litellm/pull/29612)
- fix(realtime): allow null transcripts in stream logging payloads by [@​milan-berri](https://github.com/milan-berri) in [#​29625](https://github.com/BerriAI/litellm/pull/29625)
- build(ui): migrate eslint to flat config + bump eslint-config-next to 16 by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29626](https://github.com/BerriAI/litellm/pull/29626)
- fix(key\_generate): scope session-token team-key budget exemption to caller-supplied team\_id by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29641](https://github.com/BerriAI/litellm/pull/29641)
- fix(proxy): disable proxy buffering on streaming SSE responses by [@​mateo-berri](https://github.com/mateo-berri) in [#​29557](https://github.com/BerriAI/litellm/pull/29557)
- fix(mcp): gate /public/mcp\_hub strictly on litellm.public\_mcp\_servers by [@​michelligabriele](https://github.com/michelligabriele) in [#​27764](https://github.com/BerriAI/litellm/pull/27764)
- ci(ui): frontend-lint job enforcing prettier + eslint on changed files by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29633](https://github.com/BerriAI/litellm/pull/29633)
- fix(gemini): googleSearch + server-side tools and googleMaps JSON schema by [@​Sameerlite](https://github.com/Sameerlite) in [#​29582](https://github.com/BerriAI/litellm/pull/29582)
- fix(proxy): passthrough 404 when SERVER\_ROOT\_PATH is set by [@​Sameerlite](https://github.com/Sameerlite) in [#​29658](https://github.com/BerriAI/litellm/pull/29658)
- fix(gemini-realtime): use GA event names for Pipecat 1.3.x compatibility by [@​Sameerlite](https://github.com/Sameerlite) in [#​29662](https://github.com/BerriAI/litellm/pull/29662)
- Litellm oss staging 040626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​29671](https://github.com/BerriAI/litellm/pull/29671)
- style(ui): prettier formatting pass over the dashboard by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29622](https://github.com/BerriAI/litellm/pull/29622)
- chore: ignore prettier dashboard reformat in git blame by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29695](https://github.com/BerriAI/litellm/pull/29695)
- fix(helm): Enable Backend Deployment to mount Gateway config.yaml by [@​tin-berri](https://github.com/tin-berri) in [#​29605](https://github.com/BerriAI/litellm/pull/29605)
- \[internal copy of [#​29277](https://github.com/BerriAI/litellm/issues/29277)] fix(proxy): add default=None to LiteLLM\_TeamMembership.litellm\_budget\_table by [@​mateo-berri](https://github.com/mateo-berri) in [#​29684](https://github.com/BerriAI/litellm/pull/29684)
- test: make custom\_tokenizer proxy tests hermetic by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29643](https://github.com/BerriAI/litellm/pull/29643)
- test(proxy): stop running real-DB tests in GitHub Actions unit jobs by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29700](https://github.com/BerriAI/litellm/pull/29700)
- chore(ui): remove the bare-fetch lint rule by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29712](https://github.com/BerriAI/litellm/pull/29712)
- Litellm jwt mapping virtualkeys by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​28510](https://github.com/BerriAI/litellm/pull/28510)
- refactor(ui): shared HTTP client + location-pinned fetch() lint rule by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29723](https://github.com/BerriAI/litellm/pull/29723)
- fix(proxy): stop team BYOK model name corruption on model edit by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29731](https://github.com/BerriAI/litellm/pull/29731)
- \[internal copy of [#​29511](https://github.com/BerriAI/litellm/issues/29511)] feat(guardrails): add sensitive data routing to on-premise models by [@​mateo-berri](https://github.com/mateo-berri) in [#​29531](https://github.com/BerriAI/litellm/pull/29531)
- fix(proxy/hooks): populate llm\_provider on internal rate-limit errors by [@​mateo-berri](https://github.com/mateo-berri) in [#​27707](https://github.com/BerriAI/litellm/pull/27707)
- fix(vertex/anthropic): handle namespace tools and strip client\_metadata for codex compatibility by [@​Sameerlite](https://github.com/Sameerlite) in [#​29489](https://github.com/BerriAI/litellm/pull/29489)
- Support OAuth M2M for Databricks Apps A2A agents by [@​mateo-berri](https://github.com/mateo-berri) in [#​29586](https://github.com/BerriAI/litellm/pull/29586)
- fix: small CLAUDE.md nit by [@​mateo-berri](https://github.com/mateo-berri) in [#​29749](https://github.com/BerriAI/litellm/pull/29749)
- fix(anthropic): route Claude Opus 4.8 through adaptive thinking by [@​mateo-berri](https://github.com/mateo-berri) in [#​29702](https://github.com/BerriAI/litellm/pull/29702)
- fix(proxy): persist oauth2\_flow on MCP server registration by [@​michelligabriele](https://github.com/michelligabriele) in [#​29690](https://github.com/BerriAI/litellm/pull/29690)
- \[internal copy of [#​27491](https://github.com/BerriAI/litellm/issues/27491)] fix(realtime): Fix Realtime Audio Token Cost Tracking by [@​mateo-berri](https://github.com/mateo-berri) in [#​29722](https://github.com/BerriAI/litellm/pull/29722)
- fix(galileo): use ingest traces API and standard logging payload by [@​Sameerlite](https://github.com/Sameerlite) in [#​29651](https://github.com/BerriAI/litellm/pull/29651)
- fix(auth): expand all-team-models sentinel in can\_key\_call\_model for batch validation by [@​Sameerlite](https://github.com/Sameerlite) in [#​29746](https://github.com/BerriAI/litellm/pull/29746)
- test(vcr): stop refreshing cassette TTL on read so cassettes lapse after 24h by [@​mateo-berri](https://github.com/mateo-berri) in [#​29784](https://github.com/BerriAI/litellm/pull/29784)
- test(ci): record/replay OpenAI image gen so the spend E2E isn't outage-bound by [@​mateo-berri](https://github.com/mateo-berri) in [#​29787](https://github.com/BerriAI/litellm/pull/29787)
- fix(ui): route MCP playground auth by oauth2 mode instead of token\_url by [@​tin-berri](https://github.com/tin-berri) in [#​29714](https://github.com/BerriAI/litellm/pull/29714)
- refactor(ui): centralize proxy base URL resolution into tested resolver by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29793](https://github.com/BerriAI/litellm/pull/29793)
- Litellm oss staging 050626 by [@​Sameerlite](https://github.com/Sameerlite) in [#​29774](https://github.com/BerriAI/litellm/pull/29774)
- test(google): add google-genai SDK proxy integration tests by [@​Sameerlite](https://github.com/Sameerlite) in [#​29781](https://github.com/BerriAI/litellm/pull/29781)
- fix(jwt): use resolved DB user\_id for spend on legacy email match by [@​milan-berri](https://github.com/milan-berri) in [#​29217](https://github.com/BerriAI/litellm/pull/29217)
- feat(ui): generate dashboard API types from the proxy OpenAPI spec by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29816](https://github.com/BerriAI/litellm/pull/29816)
- fix(proxy): drop deleted team BYOK model name from team.models by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29820](https://github.com/BerriAI/litellm/pull/29820)
- feat(mcp): per-server env vars with global + per-user scopes by [@​mateo-berri](https://github.com/mateo-berri) in [#​28917](https://github.com/BerriAI/litellm/pull/28917)
- refactor(ui): route behavior-preserving networking calls through apiClient by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29806](https://github.com/BerriAI/litellm/pull/29806)
- fix(mcp): persist Tools-tab MCP OAuth token to DB by [@​tin-berri](https://github.com/tin-berri) in [#​29809](https://github.com/BerriAI/litellm/pull/29809)
- fix(ui): require new expiration when regenerating an expired key by [@​milan-berri](https://github.com/milan-berri) in [#​29838](https://github.com/BerriAI/litellm/pull/29838)
- refactor(ui): route query-building networking calls through apiClient by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29815](https://github.com/BerriAI/litellm/pull/29815)
- Make the image-gen record/replay proxy report cache mode and per-request HIT/MISS by [@​mateo-berri](https://github.com/mateo-berri) in [#​29802](https://github.com/BerriAI/litellm/pull/29802)
- feat(proxy): hot-reload .env in dev when running with --reload by [@​mateo-berri](https://github.com/mateo-berri) in [#​29783](https://github.com/BerriAI/litellm/pull/29783)
- fix(ui): stop MCP playground tool calls from sending twice by [@​tin-berri](https://github.com/tin-berri) in [#​29821](https://github.com/BerriAI/litellm/pull/29821)
- feat(fal\_ai): add Nano Banana / Gemini 2.5 Flash Image generation support by [@​mateo-berri](https://github.com/mateo-berri) in [#​29798](https://github.com/BerriAI/litellm/pull/29798)
- Title: Fix managed batch cancel credential resolution by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​29734](https://github.com/BerriAI/litellm/pull/29734)
- Title: fix(proxy): resolve vector store file list credentials from team deployments by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​29739](https://github.com/BerriAI/litellm/pull/29739)
- refactor: convert AWS and GCP Terraform stacks into reusable modules … by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​28103](https://github.com/BerriAI/litellm/pull/28103)
- chore(ui): build ui for release by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29853](https://github.com/BerriAI/litellm/pull/29853)
- fix(terraform/gcp): prompt for image\_registry in DeployStack one-click by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29852](https://github.com/BerriAI/litellm/pull/29852)
- fix(terraform/gcp): abandon SQL user on destroy by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29855](https://github.com/BerriAI/litellm/pull/29855)
- Extend the record/replay proxy to chat, embeddings, moderations, rerank, and Anthropic by [@​mateo-berri](https://github.com/mateo-berri) in [#​29847](https://github.com/BerriAI/litellm/pull/29847)
- chore(deps): bump deps by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29860](https://github.com/BerriAI/litellm/pull/29860)
- chore(ci): promote internal staging to main by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29861](https://github.com/BerriAI/litellm/pull/29861)
- fix: 400 on Anthropic context overflow; seed identity on failed auth by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29848](https://github.com/BerriAI/litellm/pull/29848)
- chore(ci): promote internal staging to main by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29862](https://github.com/BerriAI/litellm/pull/29862)
- chore(release): patch v1.89.0-rc.1 with [#​30064](https://github.com/BerriAI/litellm/issues/30064) (Claude Fable 5) for v1.89.0-rc.2 by [@​mateo-berri](https://github.com/mateo-berri) in [#​30143](https://github.com/BerriAI/litellm/pull/30143)
**Full Changelog**: <https://github.com/BerriAI/litellm/compare/v1.88.0...v1.89.0>
### [`v1.88.2`](https://github.com/BerriAI/litellm/releases/tag/v1.88.2)
[Compare Source](https://github.com/BerriAI/litellm/compare/v1.88.1...v1.88.2)
#### Verify Docker Image Signature
All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0).
**Verify using the pinned commit hash (recommended):**
A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:
```bash
cosign verify \
--key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
ghcr.io/berriai/litellm:v1.88.2
```
**Verify using the release tag (convenience):**
Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:
```bash
cosign verify \
--key https://raw.githubusercontent.com/BerriAI/litellm/v1.88.2/cosign.pub \
ghcr.io/berriai/litellm:v1.88.2
```
Expected output:
```
The following checks were performed on each of these signatures:
- The cosign claims were validated
- The signatures were verified against the specified public key
```
***
#### What's Changed
- chore(release): backport Fable 5, batch-file auth, CrowdStrike AIDR, Mantle Responses SigV4, and NetApp streaming-cost fix to stable/1.88.x and cut 1.88.2 by [@​mateo-berri](https://github.com/mateo-berri) in [#​30144](https://github.com/BerriAI/litellm/pull/30144)
- chore(release): backport DB-resilience, passthrough, model-info, budget, and deps fixes to stable/1.88.x by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​30408](https://github.com/BerriAI/litellm/pull/30408)
**Full Changelog**: <https://github.com/BerriAI/litellm/compare/v1.88.1...v1.88.2>
### [`v1.88.1`](https://github.com/BerriAI/litellm/releases/tag/v1.88.1)
[Compare Source](https://github.com/BerriAI/litellm/compare/v1.88.0...v1.88.1)
#### Verify Docker Image Signature
All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0).
**Verify using the pinned commit hash (recommended):**
A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:
```bash
cosign verify \
--key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
ghcr.io/berriai/litellm:v1.88.1
```
**Verify using the release tag (convenience):**
Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:
```bash
cosign verify \
--key https://raw.githubusercontent.com/BerriAI/litellm/v1.88.1/cosign.pub \
ghcr.io/berriai/litellm:v1.88.1
```
Expected output:
```
The following checks were performed on each of these signatures:
- The cosign claims were validated
- The signatures were verified against the specified public key
```
***
#### What's Changed
- build(deps): bump pyjwt to 2.13.0 and ws override to 8.20.1 (1.88.x) by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29987](https://github.com/BerriAI/litellm/pull/29987)
- chore(release): bump version to 1.88.1 by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29989](https://github.com/BerriAI/litellm/pull/29989)
**Full Changelog**: <https://github.com/BerriAI/litellm/compare/v1.88.0...v1.88.1>
### [`v1.88.0`](https://github.com/BerriAI/litellm/releases/tag/v1.88.0)
[Compare Source](https://github.com/BerriAI/litellm/compare/v1.87.3...v1.88.0)
#### Verify Docker Image Signature
All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0).
**Verify using the pinned commit hash (recommended):**
A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:
```bash
cosign verify \
--key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \
ghcr.io/berriai/litellm:v1.88.0
```
**Verify using the release tag (convenience):**
Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:
```bash
cosign verify \
--key https://raw.githubusercontent.com/BerriAI/litellm/v1.88.0/cosign.pub \
ghcr.io/berriai/litellm:v1.88.0
```
Expected output:
```
The following checks were performed on each of these signatures:
- The cosign claims were validated
- The signatures were verified against the specified public key
```
***
#### What's Changed
- fix(proxy): gate team allowed\_passthrough\_routes to proxy admins by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​28097](https://github.com/BerriAI/litellm/pull/28097)
- fix(tests): stabilize image-edit VCR cassettes to stop live gpt-image-1 spend by [@​mateo-berri](https://github.com/mateo-berri) in [#​28110](https://github.com/BerriAI/litellm/pull/28110)
- fix(bedrock/cohere): send embedding\_types as JSON array, not string by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​28172](https://github.com/BerriAI/litellm/pull/28172)
- fix(tests): migrate realtime + rerank tests off shut-down upstream models by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28191](https://github.com/BerriAI/litellm/pull/28191)
- fix(caching): replay openai/responses bridge cache hits as chat streams by [@​Sameerlite](https://github.com/Sameerlite) in [#​28158](https://github.com/BerriAI/litellm/pull/28158)
- Litellm oss staging by [@​Sameerlite](https://github.com/Sameerlite) in [#​28161](https://github.com/BerriAI/litellm/pull/28161)
- feat(prometheus): add user\_email and user\_alias to user budget metrics by [@​Sameerlite](https://github.com/Sameerlite) in [#​28155](https://github.com/BerriAI/litellm/pull/28155)
- test(callbacks): harden flaky proxy callback-leak detector by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28195](https://github.com/BerriAI/litellm/pull/28195)
- fix(bedrock): sanitize batch metadata to prevent Pydantic ValidationError by [@​mateo-berri](https://github.com/mateo-berri) in [#​28202](https://github.com/BerriAI/litellm/pull/28202)
- fix(deepseek): use native /anthropic/v1/messages endpoint and sanitize tools by [@​mateo-berri](https://github.com/mateo-berri) in [#​28200](https://github.com/BerriAI/litellm/pull/28200)
- feat(ui): add Interactions API endpoint to playground with SSE streaming by [@​Sameerlite](https://github.com/Sameerlite) in [#​28156](https://github.com/BerriAI/litellm/pull/28156)
- fix(proxy): decode bytes and pass-through SSE for Google-native streamGenerateContent ([#​27444](https://github.com/BerriAI/litellm/issues/27444)) by [@​Sameerlite](https://github.com/Sameerlite) in [#​28213](https://github.com/BerriAI/litellm/pull/28213)
- refactor(bedrock/sagemaker): switch to lazy loading for response stre… by [@​harish-berri](https://github.com/harish-berri) in [#​28189](https://github.com/BerriAI/litellm/pull/28189)
- \[Refactor] UI - Spend Logs: consolidate filter state and extract components by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​25847](https://github.com/BerriAI/litellm/pull/25847)
- fix(tests): replace shut-down gpt-4o-audio-preview with gpt-audio-1.5 by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28281](https://github.com/BerriAI/litellm/pull/28281)
- chore(ci): bump versions by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28287](https://github.com/BerriAI/litellm/pull/28287)
- feat: propagate team\_id and team\_alias to all child OTEL spans by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​28273](https://github.com/BerriAI/litellm/pull/28273)
- Day 0 support : Gemini 3.5 Flash by [@​Sameerlite](https://github.com/Sameerlite) in [#​28268](https://github.com/BerriAI/litellm/pull/28268)
- Gemini managed agents support by [@​Sameerlite](https://github.com/Sameerlite) in [#​28270](https://github.com/BerriAI/litellm/pull/28270)
- chore(ci): promote internal staging to main by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28292](https://github.com/BerriAI/litellm/pull/28292)
- feat(gemini): add gemini-3.1-flash-lite model cost map by [@​Sameerlite](https://github.com/Sameerlite) in [#​28320](https://github.com/BerriAI/litellm/pull/28320)
- fix(spend\_counter): seed Redis counter via SET NX to prevent cross-pod double-seed by [@​milan-berri](https://github.com/milan-berri) in [#​27854](https://github.com/BerriAI/litellm/pull/27854)
- fix(proxy): normalize batch file IDs before ManagedObjectTable write by [@​Sameerlite](https://github.com/Sameerlite) in [#​28339](https://github.com/BerriAI/litellm/pull/28339)
- fix(router): use forwarded model\_id for native Azure container IDs by [@​Sameerlite](https://github.com/Sameerlite) in [#​27921](https://github.com/BerriAI/litellm/pull/27921)
- fix(ui): restore log filter loading indicator by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​28282](https://github.com/BerriAI/litellm/pull/28282)
- test(e2e): migrate runner to uv, add All Proxy Models key test by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​28313](https://github.com/BerriAI/litellm/pull/28313)
- feat(ui): team passthrough routes create parity + edit load fix by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​28098](https://github.com/BerriAI/litellm/pull/28098)
- fix(mcp): JWT on tools/list and REST tools/call server resolution by [@​Sameerlite](https://github.com/Sameerlite) in [#​28227](https://github.com/BerriAI/litellm/pull/28227)
- feat(interactions): migrate to Google Interactions API steps schema (May 2026) by [@​Sameerlite](https://github.com/Sameerlite) in [#​28153](https://github.com/BerriAI/litellm/pull/28153)
- test(ui-e2e): admin key creation with a specific proxy model by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​28365](https://github.com/BerriAI/litellm/pull/28365)
- fix(vertex\_ai): omit function\_call id on Vertex Gemini 3.5+ tool turns by [@​Sameerlite](https://github.com/Sameerlite) in [#​28324](https://github.com/BerriAI/litellm/pull/28324)
- feat(mcp): allow native MCP OAuth support for cursor by [@​Sameerlite](https://github.com/Sameerlite) in [#​28327](https://github.com/BerriAI/litellm/pull/28327)
- fix(interactions): never drop streamed text deltas; always emit terminal completion by [@​mateo-berri](https://github.com/mateo-berri) in [#​28394](https://github.com/BerriAI/litellm/pull/28394)
- fix(proxy): expose Prisma idle/connect timeout + extra DB URL params by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​28395](https://github.com/BerriAI/litellm/pull/28395)
- Litellm oss staging 1 by [@​Sameerlite](https://github.com/Sameerlite) in [#​28337](https://github.com/BerriAI/litellm/pull/28337)
- fix: serialize guardrail\_response to JSON in OTEL traces by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​28362](https://github.com/BerriAI/litellm/pull/28362)
- chore(ci): merge dev branch by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28314](https://github.com/BerriAI/litellm/pull/28314)
- test(realtime): expect session.created as xAI realtime initial event by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28424](https://github.com/BerriAI/litellm/pull/28424)
- feat(tests): behavior-pinning harness + Key Tier-1 matrix by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28321](https://github.com/BerriAI/litellm/pull/28321)
- fix(proxy): hydrate wildcard discovery credentials ([#​28284](https://github.com/BerriAI/litellm/issues/28284)) - CCI Run by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28419](https://github.com/BerriAI/litellm/pull/28419)
- Litellm oss staging 04 21 2026 2 by [@​Sameerlite](https://github.com/Sameerlite) in [#​26569](https://github.com/BerriAI/litellm/pull/26569)
- chore(ci): merge dev branch by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28290](https://github.com/BerriAI/litellm/pull/28290)
- fix(vertex\_gemma): strip `context_management` from request body by [@​mateo-berri](https://github.com/mateo-berri) in [#​28438](https://github.com/BerriAI/litellm/pull/28438)
- fix(logging): recalculate cost after router retry failures by [@​milan-berri](https://github.com/milan-berri) in [#​28476](https://github.com/BerriAI/litellm/pull/28476)
- fix(otel): emit guardrail span on violation, surface status + categories by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​28364](https://github.com/BerriAI/litellm/pull/28364)
- test(proxy): behavior-pinning matrix for team management endpoints by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28441](https://github.com/BerriAI/litellm/pull/28441)
- test(vertex\_ai): tolerate transient 500 in google maps grounding test by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28503](https://github.com/BerriAI/litellm/pull/28503)
- fix(docker): restore npm to non\_root builder image by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28519](https://github.com/BerriAI/litellm/pull/28519)
- chore(ci): bump deps by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28524](https://github.com/BerriAI/litellm/pull/28524)
- build(deps-dev): bump black to 26.3.1 and apply formatting by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28525](https://github.com/BerriAI/litellm/pull/28525)
- chore(deps): bump deps by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28528](https://github.com/BerriAI/litellm/pull/28528)
- test(e2e): forward LITELLM\_LICENSE to UI e2e proxy by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​28398](https://github.com/BerriAI/litellm/pull/28398)
- Add granian as a ASGI compliant web server. Provider better throughput stability, by [@​harish-berri](https://github.com/harish-berri) in [#​26027](https://github.com/BerriAI/litellm/pull/26027)
- Fix conflicts and UI by [@​Sameerlite](https://github.com/Sameerlite) in [#​28477](https://github.com/BerriAI/litellm/pull/28477)
- Add error\_description and hint for oauth flows by [@​Sameerlite](https://github.com/Sameerlite) in [#​28471](https://github.com/BerriAI/litellm/pull/28471)
- feat(mcp): Add tool call and tool list support via UI for Oauth mcps by [@​Sameerlite](https://github.com/Sameerlite) in [#​28454](https://github.com/BerriAI/litellm/pull/28454)
- feat(proxy): persist allowlisted OIDC claims in CLI SSO poll by [@​Sameerlite](https://github.com/Sameerlite) in [#​28463](https://github.com/BerriAI/litellm/pull/28463)
- fix(responses): use OpenAI SSEDecoder for Responses API streaming by [@​Sameerlite](https://github.com/Sameerlite) in [#​28566](https://github.com/BerriAI/litellm/pull/28566)
- Litellm oss staging 2 by [@​Sameerlite](https://github.com/Sameerlite) in [#​28582](https://github.com/BerriAI/litellm/pull/28582)
- \[internal copy of [#​28269](https://github.com/BerriAI/litellm/issues/28269)] Codex cli jwt team alias by [@​mateo-berri](https://github.com/mateo-berri) in [#​28621](https://github.com/BerriAI/litellm/pull/28621)
- fix(check\_licenses): read PEP 639 license-expression metadata by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28529](https://github.com/BerriAI/litellm/pull/28529)
- test(proxy): behavior-pinning matrix for tier-2/3 key + team management endpoints by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28620](https://github.com/BerriAI/litellm/pull/28620)
- chore(test): remove dead old Playwright e2e suite by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​28632](https://github.com/BerriAI/litellm/pull/28632)
- fix(sagemaker): send native Cohere embed payload to Cohere SageMaker endpoints by [@​milan-berri](https://github.com/milan-berri) in [#​28613](https://github.com/BerriAI/litellm/pull/28613)
- style: apply black formatting to fix lint CI (LIT-3274) ([#​28639](https://github.com/BerriAI/litellm/issues/28639)) by [@​krrish-berri-2](https://github.com/krrish-berri-2) in [#​28641](https://github.com/BerriAI/litellm/pull/28641)
- fix(bedrock): decouple STS region from Bedrock aws\_region\_name by [@​milan-berri](https://github.com/milan-berri) in [#​28245](https://github.com/BerriAI/litellm/pull/28245)
- test(streaming): tolerate Vertex 429 wrapped in MidStreamFallbackError by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28669](https://github.com/BerriAI/litellm/pull/28669)
- feat(guardrails): add Microsoft Purview DLP guardrail by [@​Sameerlite](https://github.com/Sameerlite) in [#​24966](https://github.com/BerriAI/litellm/pull/24966)
- fix(mcp): forward upstream initialize instructions on cold gateway init by [@​milan-berri](https://github.com/milan-berri) in [#​28231](https://github.com/BerriAI/litellm/pull/28231)
- chore(ci): promote internal staging to main by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28680](https://github.com/BerriAI/litellm/pull/28680)
- CI: copy of [#​25177](https://github.com/BerriAI/litellm/issues/25177) (OCI GenAI: embeddings, streaming/reasoning fixes, model catalog) by [@​mateo-berri](https://github.com/mateo-berri) in [#​28223](https://github.com/BerriAI/litellm/pull/28223)
- Encrypt callback\_vars in key/team metadata in DB by [@​Michael-RZ-Berri](https://github.com/Michael-RZ-Berri) in [#​27141](https://github.com/BerriAI/litellm/pull/27141)
- perf: reduce per-request and per-chunk overhead across Anthropic streaming hot paths by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​28289](https://github.com/BerriAI/litellm/pull/28289)
- feat(azure): add Speech STT config support by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​27482](https://github.com/BerriAI/litellm/pull/27482)
- test(proxy): phase-4 payload behavior pinning for tier-2/3 key + team management endpoints by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28681](https://github.com/BerriAI/litellm/pull/28681)
- feat(prometheus): emit per-token-type detail metrics (LIT-3220) ([#​28372](https://github.com/BerriAI/litellm/issues/28372)) by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​28378](https://github.com/BerriAI/litellm/pull/28378)
- fix(otel): stamp http.response.status\_code on all error responses by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​28405](https://github.com/BerriAI/litellm/pull/28405)
- chore(ui): build ui by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28707](https://github.com/BerriAI/litellm/pull/28707)
- fix(helm): drop main- prefix from default image tag by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28710](https://github.com/BerriAI/litellm/pull/28710)
- test(model\_prices): allow audio\_transcription\_config in schema by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28708](https://github.com/BerriAI/litellm/pull/28708)
- chore(ci): promote internal staging to main by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28709](https://github.com/BerriAI/litellm/pull/28709)
- fix(team): refresh team cache on team\_model\_add/delete (LIT-3244) by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28683](https://github.com/BerriAI/litellm/pull/28683)
- fix(ui/add-model): stop vertex\_ai-anthropic\_models from leaking into Anthropic dropdown by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​28723](https://github.com/BerriAI/litellm/pull/28723)
- Fix spend logs v2 route permissions by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​28705](https://github.com/BerriAI/litellm/pull/28705)
- fix(proxy): Bedrock Knowledge Base pass-through: preserve SigV4 headers and signed request body by [@​milan-berri](https://github.com/milan-berri) in [#​27526](https://github.com/BerriAI/litellm/pull/27526)
- chore(tests): migrate Bedrock CI to AWS account [`9412775`](https://github.com/BerriAI/litellm/commit/941277531214) by [@​mateo-berri](https://github.com/mateo-berri) in [#​28728](https://github.com/BerriAI/litellm/pull/28728)
- fix(otel): export SERVER span on management-endpoint success without http\_request by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​28794](https://github.com/BerriAI/litellm/pull/28794)
- chore(ci): merge dev branch by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28801](https://github.com/BerriAI/litellm/pull/28801)
- chore(ci): merge dev branch by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28657](https://github.com/BerriAI/litellm/pull/28657)
- fix(ui): show 2-decimal precision for max\_budget on key overview by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​28809](https://github.com/BerriAI/litellm/pull/28809)
- feat(proxy): allow `llm_api_routes` virtual keys to list MCP servers by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​28442](https://github.com/BerriAI/litellm/pull/28442)
- chore(ci): merge dev branch by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28807](https://github.com/BerriAI/litellm/pull/28807)
- fix(team): keep team\_alias cache in sync on \_cache\_team\_object writes by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28737](https://github.com/BerriAI/litellm/pull/28737)
- chore(ci): merge dev branch by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28822](https://github.com/BerriAI/litellm/pull/28822)
- ci: daily oss-agent-shin canonical branch by [@​ishaan-berri](https://github.com/ishaan-berri) in [#​28829](https://github.com/BerriAI/litellm/pull/28829)
- test(proxy): add harness for proxy\_server.py behavior-pinning by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28827](https://github.com/BerriAI/litellm/pull/28827)
- feat(openai): apply regional-processing cost uplift for EU/US data residency by [@​mateo-berri](https://github.com/mateo-berri) in [#​28626](https://github.com/BerriAI/litellm/pull/28626)
- chore(admin-ui): regenerate static export with trailingSlash: true by [@​mateo-berri](https://github.com/mateo-berri) in [#​28112](https://github.com/BerriAI/litellm/pull/28112)
- fix(azure): preserve AD token refresh in v1 OpenAI client path by [@​mateo-berri](https://github.com/mateo-berri) in [#​28627](https://github.com/BerriAI/litellm/pull/28627)
- fix(ui): route API Reference back to query-param page by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​28726](https://github.com/BerriAI/litellm/pull/28726)
- fix(model-edit): allow clearing custom pricing on wildcard models by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​28719](https://github.com/BerriAI/litellm/pull/28719)
- fix(tests/vcr): make Redis cassette cache replay deterministically (zero VCR misses on consecutive runs) by [@​mateo-berri](https://github.com/mateo-berri) in [#​28826](https://github.com/BerriAI/litellm/pull/28826)
- fix(proxy): strip LiteLLM policy tracking from OpenAI batch metadata by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​28425](https://github.com/BerriAI/litellm/pull/28425)
- Litellm OpenAI double prefix bug by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​28661](https://github.com/BerriAI/litellm/pull/28661)
- Litellm oss staging 250526 by [@​Sameerlite](https://github.com/Sameerlite) in [#​28770](https://github.com/BerriAI/litellm/pull/28770)
- fix(bedrock): align toolUse/toolSpec names and allow hyphens by [@​Sameerlite](https://github.com/Sameerlite) in [#​28874](https://github.com/BerriAI/litellm/pull/28874)
- fix(realtime): send TEXT frames and valid guardrail session.update by [@​Sameerlite](https://github.com/Sameerlite) in [#​28848](https://github.com/BerriAI/litellm/pull/28848)
- fix(mcp): extend key access-group union to MCP servers by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​28890](https://github.com/BerriAI/litellm/pull/28890)
- fix(galileo): support hosted v2 spans API and string output extraction by [@​Sameerlite](https://github.com/Sameerlite) in [#​28771](https://github.com/BerriAI/litellm/pull/28771)
- fix(proxy): exclude proxy\_server\_request from its own body snapshot by [@​michelligabriele](https://github.com/michelligabriele) in [#​28618](https://github.com/BerriAI/litellm/pull/28618)
- \[Feat] Add tool calling support for gemini and vertex ai live api by [@​Sameerlite](https://github.com/Sameerlite) in [#​26590](https://github.com/BerriAI/litellm/pull/26590)
- refactor(ui): remove dead App Router scaffolding in (dashboard)/\* by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​28891](https://github.com/BerriAI/litellm/pull/28891)
- fix(docker): use system Node in componentized builders + retry apk add by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​28888](https://github.com/BerriAI/litellm/pull/28888)
- docs(agents): require consent before writing new third-party names by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​28908](https://github.com/BerriAI/litellm/pull/28908)
- refactor(ui): extract auth state into AuthContext by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​28910](https://github.com/BerriAI/litellm/pull/28910)
- fix(mcp): resolve team.access\_group\_ids → MCP servers by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​28997](https://github.com/BerriAI/litellm/pull/28997)
- test(ui): e2e cover team model edit + admin identity in navbar by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​28652](https://github.com/BerriAI/litellm/pull/28652)
- test(e2e): cover add-fallback flow in Router Settings by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29069](https://github.com/BerriAI/litellm/pull/29069)
- test(e2e): cover Team-BYOK add-model flow as proxy admin by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29068](https://github.com/BerriAI/litellm/pull/29068)
- fix(containers): record ownership for service-account keys + fix Prisma Json serialization by [@​Sameerlite](https://github.com/Sameerlite) in [#​28990](https://github.com/BerriAI/litellm/pull/28990)
- test(e2e): cover add-MCP-server flow via discovery → custom form by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29070](https://github.com/BerriAI/litellm/pull/29070)
- test(e2e): cover AI Hub make-public flow and public model\_hub\_table by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29071](https://github.com/BerriAI/litellm/pull/29071)
- \[internal copy of [#​28877](https://github.com/BerriAI/litellm/issues/28877)] feat: add support for claude code goal mode for bedrock opus output config by [@​mateo-berri](https://github.com/mateo-berri) in [#​28898](https://github.com/BerriAI/litellm/pull/28898)
- feat(guardrails): wire apply\_guardrail into proxy logging callbacks by [@​Sameerlite](https://github.com/Sameerlite) in [#​28970](https://github.com/BerriAI/litellm/pull/28970)
- chore(ci): merge dev brach by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29192](https://github.com/BerriAI/litellm/pull/29192)
- perf(streaming): cut per-chunk overhead \~30% on Anthropic + Bedrock hot path by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​28720](https://github.com/BerriAI/litellm/pull/28720)
- fix(proxy): enforce tag budgets for key-level tags by [@​Sameerlite](https://github.com/Sameerlite) in [#​29108](https://github.com/BerriAI/litellm/pull/29108)
- fix(vertex-ai): use DB credentials in video handlers + implement Veo video edit by [@​Sameerlite](https://github.com/Sameerlite) in [#​29098](https://github.com/BerriAI/litellm/pull/29098)
- fix(datadog): drain cost-management queue + opt-in FinOps tag allowlist by [@​michelligabriele](https://github.com/michelligabriele) in [#​28487](https://github.com/BerriAI/litellm/pull/28487)
- feat(helm): split per-component ServiceAccounts for gateway, backend, and UI by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​28712](https://github.com/BerriAI/litellm/pull/28712)
- chore(ci): bump deps ([#​29208](https://github.com/BerriAI/litellm/issues/29208)) by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29226](https://github.com/BerriAI/litellm/pull/29226)
- fix(tests/vcr): mint Google OAuth tokens live to prevent stale-token replay by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29229](https://github.com/BerriAI/litellm/pull/29229)
- chore(cookbook): bump Go directive to 1.26.3 in gollem example by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29234](https://github.com/BerriAI/litellm/pull/29234)
- chore(ci): bump version by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29242](https://github.com/BerriAI/litellm/pull/29242)
- feat(anthropic): add Claude Opus 4.8 and prune reasoning-effort flags by [@​mateo-berri](https://github.com/mateo-berri) in [#​29238](https://github.com/BerriAI/litellm/pull/29238)
- chore(ci): promote internal staging to main by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29243](https://github.com/BerriAI/litellm/pull/29243)
- fix(ci): restore real Bedrock batch S3 bucket/role in oai\_misc\_config by [@​mateo-berri](https://github.com/mateo-berri) in [#​29245](https://github.com/BerriAI/litellm/pull/29245)
- fix(guardrails): persist disable\_global\_guardrails on keys by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29233](https://github.com/BerriAI/litellm/pull/29233)
- test(e2e): cover Team Admin view + member + key flows by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29072](https://github.com/BerriAI/litellm/pull/29072)
- docs: hand-written CLAUDE.md; remove AGENTS.md, point GEMINI.md at it by [@​mateo-berri](https://github.com/mateo-berri) in [#​29252](https://github.com/BerriAI/litellm/pull/29252)
- fix(teams): expose keys\_count on /v2/team/list and wire UI Resources badge by [@​michelligabriele](https://github.com/michelligabriele) in [#​28502](https://github.com/BerriAI/litellm/pull/28502)
- fix(anthropic): stop injecting unsupported output\_config.effort=xhigh for Claude Code on Sonnet/Opus 4.6 by [@​mateo-berri](https://github.com/mateo-berri) in [#​29304](https://github.com/BerriAI/litellm/pull/29304)
- test(e2e): cover Internal Viewer nav, key, and team-info gating by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29075](https://github.com/BerriAI/litellm/pull/29075)
- test(e2e): cover Internal User key modal, team info, key page by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29074](https://github.com/BerriAI/litellm/pull/29074)
- test(e2e): cover navbar Logout flow as proxy admin by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29076](https://github.com/BerriAI/litellm/pull/29076)
- fix(mcp): resolve key.access\_group\_ids → MCP servers (ungated) by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29195](https://github.com/BerriAI/litellm/pull/29195)
- fix(router): enforce deployment budgets for dynamically added models by [@​Sameerlite](https://github.com/Sameerlite) in [#​29273](https://github.com/BerriAI/litellm/pull/29273)
- fix(proxy): map stripped batch body.model to proxy alias for auth by [@​Sameerlite](https://github.com/Sameerlite) in [#​29264](https://github.com/BerriAI/litellm/pull/29264)
- feat(mcp): support stateless and stateful clients via session-id routing by [@​Sameerlite](https://github.com/Sameerlite) in [#​26857](https://github.com/BerriAI/litellm/pull/26857)
- fix(bedrock): support tool search results + chat annotations by [@​Sameerlite](https://github.com/Sameerlite) in [#​29120](https://github.com/BerriAI/litellm/pull/29120)
- fix(mcp): ignore stale ids on key save by [@​Sameerlite](https://github.com/Sameerlite) in [#​29128](https://github.com/BerriAI/litellm/pull/29128)
- feat(a2a): well-known agent-card discovery + LangGraph Platform mode by [@​Sameerlite](https://github.com/Sameerlite) in [#​28860](https://github.com/BerriAI/litellm/pull/28860)
- fix(proxy): link passthrough success spans to the SERVER root OTEL span by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29315](https://github.com/BerriAI/litellm/pull/29315)
- \[internal copy of [#​29089](https://github.com/BerriAI/litellm/issues/29089)] fix: duplicate claude code traces by [@​mateo-berri](https://github.com/mateo-berri) in [#​29311](https://github.com/BerriAI/litellm/pull/29311)
- feat(otel): typed semconv-aligned OpenTelemetry instrumentation by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​28909](https://github.com/BerriAI/litellm/pull/28909)
- tests(proxy\_server): surface current behavior in tests by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29309](https://github.com/BerriAI/litellm/pull/29309)
- test(e2e): cover Internal User create-key flow when in no teams by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29083](https://github.com/BerriAI/litellm/pull/29083)
- test(e2e): assert internal-user navbar identity is scoped to that user by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29077](https://github.com/BerriAI/litellm/pull/29077)
- feat(otel): add team\_metadata, http.route, and model names to inference spans by [@​yassin-berriai](https://github.com/yassin-berriai) in [#​29319](https://github.com/BerriAI/litellm/pull/29319)
- feat(context\_management): compact\_20260112 polyfill for non-Anthropic providers by [@​Sameerlite](https://github.com/Sameerlite) in [#​28868](https://github.com/BerriAI/litellm/pull/28868)
- feat(enterprise): add RESEND\_FROM\_EMAIL for self-hosted Resend sends by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​28830](https://github.com/BerriAI/litellm/pull/28830)
- Revert Bedrock CI back to the reactivated AWS account ([`8886022`](https://github.com/BerriAI/litellm/commit/888602223428)) by [@​mateo-berri](https://github.com/mateo-berri) in [#​29326](https://github.com/BerriAI/litellm/pull/29326)
- fix(mcp): preserve source\_url in GET /v1/mcp/server list responses by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​29249](https://github.com/BerriAI/litellm/pull/29249)
- fix(mcp): preserve omitted fields on PUT /v1/mcp/server partial updates by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​29253](https://github.com/BerriAI/litellm/pull/29253)
- fix(ci): make litellm\_internal\_staging green (logging test + Bedrock Opus 4.7 self-heal) by [@​mateo-berri](https://github.com/mateo-berri) in [#​29344](https://github.com/BerriAI/litellm/pull/29344)
- refactor(proxy/auth): normalize Bearer prefix in safe-hash helper by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29343](https://github.com/BerriAI/litellm/pull/29343)
- test(reasoning-effort-grid): cover Claude Opus 4.8 across provider routes by [@​mateo-berri](https://github.com/mateo-berri) in [#​29327](https://github.com/BerriAI/litellm/pull/29327)
- fix(guardrails): return HTTP 400 for litellm content filter blocks by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​28418](https://github.com/BerriAI/litellm/pull/28418)
- fix(proxy): restrict vector store index create/delete to proxy admins by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​29202](https://github.com/BerriAI/litellm/pull/29202)
- feat(pass\_through): extend passthrough\_managed\_object\_ids to Azure by [@​Sameerlite](https://github.com/Sameerlite) in [#​29160](https://github.com/BerriAI/litellm/pull/29160)
- fix(proxy): enforce allowed\_passthrough\_routes for auth=true pass-thr… by [@​shivamrawat1](https://github.com/shivamrawat1) in [#​29256](https://github.com/BerriAI/litellm/pull/29256)
- feat(mcp/auth): additive key access-group grants + opt-in member assignment by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29313](https://github.com/BerriAI/litellm/pull/29313)
- fix(reset\_budget): write only {spend, budget\_reset\_at} and stop pre-zeroing counter by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29358](https://github.com/BerriAI/litellm/pull/29358)
- test(e2e): cover PROXY\_LOGOUT\_URL redirect on Logout by [@​ryan-crabbe-berri](https://github.com/ryan-crabbe-berri) in [#​29080](https://github.com/BerriAI/litellm/pull/29080)
- fix(ui): break logout redirect loop across dev and proxy origins by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29360](https://github.com/BerriAI/litellm/pull/29360)
- fix(openai-moderation): wire streaming flags through to unified dispatcher by [@​michelligabriele](https://github.com/michelligabriele) in [#​27324](https://github.com/BerriAI/litellm/pull/27324)
- chore(ci): build ui by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29366](https://github.com/BerriAI/litellm/pull/29366)
- fix(v3 limiter): cap no-max\_tokens TPM floor at smallest configured limit by [@​michelligabriele](https://github.com/michelligabriele) in [#​28805](https://github.com/BerriAI/litellm/pull/28805)
- fix(e2e): tolerate trailing slash in SERVER\_ROOT\_PATH login redirect by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29369](https://github.com/BerriAI/litellm/pull/29369)
- chore(deps): bump deps by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29373](https://github.com/BerriAI/litellm/pull/29373)
- chore(ci): promote internal staging to main by [@​yuneng-berri](https://github.com/yuneng-berri) in [#​29372](https://github.com/BerriAI/litellm/pull/29372)
- chore(release): patch v1.88.0-rc.1 with four staged fixes by [@​mateo-berri](https://github.com/mateo-berri) in [#​29632](https://github.com/BerriAI/litellm/pull/29632)
- chore(release): patch v1.88.0-rc.1 with [#​29612](https://github.com/BerriAI/litellm/issues/29612) (session-token budget-ceiling exemption) by [@​mateo-berri](https://github.com/mateo-berri) in [#​29637](https://github.com/BerriAI/litellm/pull/29637)
- fix(key\_generate): harden GHSA-q775 …
* feat(a2a): add watsonx Orchestrate agent provider Bridge A2A message/send to WXO runs API (CP4D and IBM Cloud IAM auth), with dashboard agent type metadata and unit tests for transformations. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): use shared httpx client and cache WXO auth tokens Route WXO streaming through get_async_httpx_client (TLS verification enabled). Cache bearer tokens with TTL buffer. Extract A2A reply text via a dedicated helper instead of hard-coded JSON paths. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): treat CP4D token expiration as absolute Unix time CP4D /authorize returns expiration as epoch seconds, not TTL. Compute remaining lifetime against wall clock so cached tokens refresh before expiry. Co-authored-by: Cursor <cursoragent@cursor.com> * style(a2a): black-format watsonx orchestrate handler for CI py312 Co-authored-by: Cursor <cursoragent@cursor.com> * Fix watsonx orchestrate edge cases * Fix WXO streaming fallback error handling * Fix watsonx orchestrate run completion handling * fix(a2a): make WXO username optional in agent create UI Username is only required for cp4d auth; ibm_cloud uses api_key alone. Backend still validates username when auth_mode is cp4d. Co-authored-by: Cursor <cursoragent@cursor.com> * test(a2a): align WXO dashboard field test with optional username Username is not required in agent_create_fields.json; backend validates for cp4d auth_mode only. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): send Accept header on WXO streaming run request * fix(a2a/wxo): scope streaming transport fallback to initial POST only Narrow the httpx.TransportError fallback in handle_streaming so it only covers the initial POST /runs/stream. Errors during polling or SSE consumption now propagate instead of triggering handle_non_streaming, which would have submitted a duplicate WXO run for the same request. * refactor(a2a/wxo): type run-param extraction and use text response_type Return a typed WXORequestParams NamedTuple from _extract_litellm_params instead of a positional tuple so call sites read params by name, and send the user message with response_type 'text' so the run body is valid across all WXO agent configurations rather than the search-specific type. * fix(a2a/wxo): evict expired token cache entries and raise asyncio.TimeoutError on poll timeout --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Summary
watsonx_orchestrateA2A provider (auth, submit run, poll, SSE + fallback streaming).A2AProviderConfigManager.agent_create_fields.json.Fix LIT-1967
Test plan
pytest tests/test_litellm/a2a_protocol/providers/watsonx_orchestrate/test_watsonx_orchestrate_transformation.py -v/a2a/{agent}/message/sendNote
Medium Risk
Introduces new outbound auth (API keys, bearer tokens) and long-running poll/stream HTTP to customer CP4D/IBM endpoints; failures are mostly isolated to the new provider but misconfiguration can leak credentials into logs if existing patterns do.
Overview
Adds a
watsonx_orchestrateA2A provider so LiteLLM can drive IBM watsonx Orchestrate agents through the existing agent/A2A path.Provider wiring:
A2AProviderConfigManagerresolvescustom_llm_provider=watsonx_orchestrateto a new config/handler stack that translates A2A send/stream requests into WXO REST calls (POST /v1/orchestrate/runs, pollGET .../runs/{id}, optionalPOST .../runs/stream), then maps results back to A2A JSON-RPC (non-streamingmessageresponses and streaming task/status/artifact events, including fake streaming when the upstream response is already complete).Auth & params: Supports CP4D (
username+ API key → cluster authorize) and IBM Cloud IAM (API key only), with in-memory bearer token caching. Requiredlitellm_params:cp4d_host,instance_id,wxo_agent_id,api_key; optionalauth_mode,thread_id,username(enforced at runtime for CP4D).Proxy UI:
agent_create_fields.jsongains a watsonx Orchestrate agent type with credential fields andcustom_llm_provider: watsonx_orchestrate.Tests: New unit tests cover transformations, token TTL, SSE accumulation, streaming poll/failure paths, config manager registration, and dashboard field defaults.
Reviewed by Cursor Bugbot for commit 900926a. Bugbot is set up for automated code reviews on this repo. Configure here.